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engagement activities proposed, planned and conducted within its Area Of 
Responsibility. USPACOM’s goals and objectives for the engagement process have been 
formally defined, but no process exists to link engagement activities to goals and 
objectives. Consequently, there 1s no way to comparatively assess the value of one 
engagement activity versus another. This research focuses on the basics of Multi- 
Attribute Utility Theory (MAUT) and Analytical Hierarchy Process (AHP) techniques for 
alternative selection following a literature review, which addresses some of the issues in 
decision support, traditional modeling techniques, and some of the traditional 
methodologies for quantifying subjective judgments. Additionally, this research 
illustrates the usefulness of currently available COTS decision support software in 
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I. INTRODUCTION 


A. BACKGROUND 
The mission of United States Pacific Command (USPACOM) is to enhance 

security and promote peaceful development in the Asia-Pacific region by deterring 
ageression, responding to crisies and fighting to win in support of our national security 
objectives. The U. S. Commission on National Security/21* Century, co-chaired by 
Senators Gary Hart and Warren Rudman (2000), recently stated that among other 1ssues: 

It 1s a critical national interest of the United States that no hostile power establish 

itself on U.S. borders, or in control of critical land, air, and sea lines of 

communication, or--in today's new world--in control of access to outer space or 

cyberspace. It 1s a critical national interest of the United States that no hostile 

hegemon arise 1n any of the globe's major regions, nor a hostile global peer rival or a 

hostile coalition comparable to a peer rival... 
Additionally, the commission stated that the security of allies and friends is a critical 
national interest of the United States and that one of the key objectives is to assist in the 
economic and political integration of China, India and Russia into the mainstream of the 
international community (Hart and Rudman, 2000). USPACOM’s Theater Engagement 
Plan (TEP) contributes to the accomplishment of this mission by planning, coordinating, 
and implementing peacetime military activities to shape the region’s security 
environment. 

The design of USPACOM’s TEP is to identify realistic endstates that support the 

prioritized regional objectives for the Area of Responsibility (AOR), plan those activities 


that support accomplishment of the approved endstates and coordinate the components’ 


efforts to shape the Asia-Pacific regional security environment. Engagement planning 


helps foster the development of security communities by focusing on three themes: 
goodwill, access, and competent coalition partners. 

USPACOM’s engagement strategy for the AOR is to enhance basic goodwill 
relations, improve access for U.S. forces and develop competent coalition partners in 
order to facilitate the development of security communities, which in-turn, will enhance 
security and promote peaceful development in the Asia-Pacific region. 

B. STATEMENT OF THE PROBLEM 

The current TEP’s guidance and assistance policy focuses on the engagement of 
Security Communities within its theater. These Security Communities are defined as 
groups of nations that don’t plan or want to fight each other, that cooperate in the 
peaceful resolution of disputes, and that work together on other common issues. These 
groups of nations are considered neither a defense community nor a security alliance. 

As previously stated, USPACOM’s engagement strategy, as it concerns these 
security communities, focuses on three entities: goodwill relations, access (training and 
contingencies), and regional readiness. Regional readiness is the mechanism for 
developing competent coalition partners and is mainly concerned with interoperability 
issues such as tactics, techniques and procedures and foreign military sales. Regional 
readiness is also concerned with combined operations involving humanitarian issues. 

The engagement strategy of each individual country focuses on four areas of 
engagement. USPACOM is interested in utilizing a coordinated focus approach in order 
to expand current engagement activities. USPACOM is also interested in continuing 


existing relationships to foster current and future engagement activities. Additionally, 


USPACOM is interested in increasing the potential efficiencies in order to tailor current 
and future engagement activities to achieve specific objectives. Finally, USPACOM is 
interested in developing a way to effectively interact wıth countries that possess 
engagement restrictions. These engagement restrictions may exist due to various 


political, military and/or economic considerations. 
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Figure 1.1 - USPACOM Engagement Planning Cycle. 


The endstates and objectives of the theater engagement activities are put forth by 
the USPACOM Commander-In-Chief (CINC) and are not consider negotiable. However, 


the ends and means of achieving these objectives are somewhat negotiable between the 


joint staff members and USPACOM’s subordinate and supporting agencies. 

The USPACOM engagement planning cycle is a 12-month cycle that involves 
inputs from numerous and various internal and external sources. As shown in Figure 1.1, 
the Engagement Working Groups (EWGs), consisting of country representatives, meets 
four times throughout the year and plan projected engagement activities for forthcoming 
fiscal years. This planning cycle consists of the coordination, interaction, and decision 
making of 35 separate internal and external staff and support entities in the pursuit of the 
USPACOM’s engagement goals and objectives and 15 projected out for seven fiscal 
years. 

The engagement activities employed by USPACOM are categorized by type and 
consist of 12 separate areas, which fall under eight major TEP categories. These major 


TEP categories are as follows: 


Operational Activities 
Exercises 

Combined Training 
Combined Education 

Military Contacts 

Security Assistance 
Humanitarian Assistance 
Other Engagement Operations 


Under these main categories, USPACOM is interested in pursuing 12 types of 
engagement activities: Counterdrug and Repatriation operations (Operational Activities); 
Exercises; Training; Education; High Level Visits, Port Visits, Conferences, and 
Exchanges (Military Contacts); Education and Foreign Military Sales (Security 


Assistance); Humanitarıan Assistance; and Other Engagement Operations. It should be 


noted that the quantity, level and type of engagement activity chosen is dependent on the 
intricacies of the particular country being considered. 

USPACOM J56 (Future Plans and Operations, Engagement) has a requirement to 
assess the effectiveness of engagement activities proposed, planned and conducted within 
USPACOM’s AOR. Although the CINC's goals and objectives for the engagement 
process have been formally defined, there exists no formal process to link the 
engagement activities and their effectiveness to the goals and objectives. Consequently, 
there currently 1s no way to gauge the impact of the most significant engagement activity 
or comparatively assess the value of one activity type versus another in the achievement 
of a particular goal or objective. 

C. SCOPE 

The aim of this work 1s to develop a basic understanding of what decision analysis 
and decision support aides can do for decision-makers, their usefulness, appropriateness, 
and some of their limitations in the context of engagement activity selection and 
evaluation. 

This thesis will begin with a limited review of decision analysis, decision models, 
and decision support systems in order to provide a basic foundation for other concepts 
presented 1n later sections. This work will also briefly look at the function and utility of 
different model types that may be useful to decision-makers in the process of decision 
analysis and alternative selection. Additionally, this thesis will provide a limited review 
and comparison of the theoretical basis of Multi-Attribute Utility Theory (MAUT) and 


the Analytical Hierarchy Process (AHP) decision analysis and support methodologies and 


how they may be applied to USPACOM’s engagement activity selection problem. 

The primary focus of this E will provide a limited analysis of the capabilities 
of Expert Choice'", a Commercial-Off-The-Shelf (COTS) software application, which 
employs the AHP methodology. The purpose of this analysis will be to determine the 
usefulness of this application in aiding USPACOM’s decision-makers in selecting the 


most effective theater engagement activities from a list of available alternatives. 


П. REVIEW OF DECISION THEORY, MODELS, AND SUPPORT 
SYSTEMS 


A. DECISION THEORY 

A good decision usually results from the application of logic, review of all 
available data and alternatives, and pursuit of a quantitative approach. There are times 
when a decision, initially thought to be good, obtains unfavorable or unexpected results. 
Although the results of the decision were unfavorable, the decision is still a good one due 
to the fact that the decision was made properly. Conversely, poor decisions that fail to 
apply logic, fail to consider all of the available information, fail to consider all of the 
alternatives, and fail to apply some quantitative approach can still result in a favorable 
outcome. However, a decision that produces favorable results through the use of faulty 
logic and incomplete data and information is still considered a poor decision. The 
application of some of the basic tenets of decision theory along with the use of some 
decision support systems will increase the occurrence in which the decision-maker 
achieves a successful outcome. (Render and Stair, 1997) 

Regardless of the type of decision to be made, Render and Stair (1997) believe 
that decision-making is a six-step process. These six steps are as follows: 
The problem is clearly defined. 
All possible alternatives are listed. 
Possible outcomes are identified. 
Each combination of alternative and outcome is listed with respect to its 
expected payoff. 


Select one of the available decision support theory models. 
Apply the model, analyze the results and make your decision. 


Би ЗЕ 
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The amount and type of ınformatıon or knowledge they have about the particular 
situation usually influences the decision-making of most leaders. In 1960, Dr. Herbert 
A. Simon provided an early descriptive model of the decision-making process that 
involved the use of three major sub-processes. These three sub-processes are just as 
sound today and still very applicable. The sub-processes are as follows: 


e Intelligence — The stage where the decision-makers scan their environment for 
potential problems/opportunities that will require the making and execution of 
decisions. 

е Design — The stage that consists of the generation, development, analysis and 
feasibility assessment of potential courses of action or alternatives. 

e Choice — The stage where one course of action or alternative is selected and 
eventually implemented. (Young, 1989) 


As proof of their validity today, compare them with Young’s (1989) statement that 
Decision theory envisions any decision problem as consisting of the elements of: 


e Alternative strategies defined as courses of action or a particular combination 
of “settings” of the variables under the control of the decision-maker. 

e A combination of conditions (particular *settings") of relevant variables 
which are not controllable by the decision-maker (traditionally called “states 
of nature” although the conditions could be man-made by persons other than 
the decision-maker). 

e Interactions between strategy and each state of nature that result in outcomes 
of importance to the decision-maker and which can be measured in some form 
of payoff units (often money). 

e А спїепоп or analysis rule by which a decision-maker can assess the situation 
and select a particular strategy. 


Regardless of the composition of the decision problem, this decision-making 
usually occurs in one of three types of environments: decision-making under certainty; 
decision-making under risk; and decision-making under uncertainty. Decision-making 
under certainty involves the occurrence where decision-makers know the consequences 


of every alternative and chooses the alternative with the most favorable outcome. 


Decision-making under risk concerns decision-makers’ knowledge of the probability of 
each alternative and their desire to maximize the expected result. Lastly, decision- 
making under uncertainty concerns the situation where decision-makers have no 
knowledge of the probabilities of the possible outcomes. (Render and Stair, 1997) 

Sage (1991) believes that a fourth environment, decision under conflict, exists and 
that it is more appropriately handled with game theory and conflict analysis. Problems 
that arise in the environment of decision-making under the conditions of certainty are 
usually addressed by deterministic decision theory. Decision analysis models are usually 
appropriate for dealing with problems associated with decision-making under risk and 
decision making under uncertainty. The selection of the best engagement activity from a 
set of alternatives is a problem associated with risk and/or uncertainty. 

B. DECISION MODELS 

This section describes the system engineering approach to problem solution and 
the modeling techniques proposed by Sage (1991). The basic guidelines of this approach 
to problem solution include: 

1. Definition of the problem or issue formulation. 

2. Analysis of the problem or issue. 

3. Interpretation of the analysis, which includes alternative evaluation and 

selection and implementation. 

Under the first guideline, the focus 1s on the definition of the problem. This 
formulation process includes identification of the individual problem elements or 
characteristics. Problem definition is normally a group activity involving those 
individuals who are most familiar with the issue at hand (e.g., engagement working 


groups). If done properly, this process identifies the needs, constraints, alterables, 


political factors, and military considerations affecting a particular problem. This process 
should also serve to 1dentify the relationships among these elements. 

An important concern in this process 1s the identification and structuring of the 
goals and objectives for the alternative that 1s eventually selected by the decision-makers. 
Using the concept of collective inquiry, there are two groups of methods for achieving 
this goal. The first group includes brainstorming, synectics and nominal group technique, 
all require that a facilitator-led group meet together in same place and at the same time. 
The nominal group techniques uses the step process of idea generation, discussion, and 
prioritization and 1s generally more effective than brainstorming in reducing the influence 
of dominate personalities. Synectics is based on problem analogies and is considered 
more appropriate for the generation of truly innovative, unconventional ideas. Also, it 
normally requires a more experienced facilitator and group. Synectics, along with 
brainstorming is considered to be directly interactive where nominal is indirectly 
interactive. Group members using the nominal group technique may not communicate 
directly. 

The second group of collective inquiry includes questionnaires, survey and 
Delphi. Although these methods do not require the gathering of the group members at a 
particular time and place, they do tend to take more time to complete. With no 
interaction among the participants, questionnaires and surveys solicit individual answers 
to questions from a large group and then derive the overall results. Delphi technique 
normally requires a written anonymous response over several rounds. The results of each 


previous round provide feedback to the participants who are asked to comment, revise 
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and/or reiterate their views. Although thıs technique may prove to be highly instructive, 
1t can be a lengthy process. 

Interaction matrices, trees, structured modeling, and casual loop diagrams are 
some of the many available element-structuring aids that can be useful dunng this step. 
Interaction matrices may be helpful in identifying clusters of closely related elements or 
identifying the couplings of elements from different sets. Trees are graphical aids that 
are useful in the portrayal of hierarchical structures. Structured modeling techniques are 
usually computer aides designed to assists individuals or groups in structuring large sets 
of elements. Casual loop diagrams provide a graphical heuristic of the casual interactions 
between sets of variables. The use of any of the available structuring methods can lead to 
greater clarity of the problem as well as the identification of new or revised descriptions 
of the problem elements. 

The second guideline proposed by Sage focuses on problem analysis and 
generally consists of two steps. The first step, impact assessment, is the analysis of the 
alternatives to determine their impact on the overall goal or objective. The second step 
involves an optimization process, which attempts to maximize the alternatives’ 
performance in terms of goal satisfaction within the stated problem constraints. 

Simulation and modeling techniques are very useful for completing this second 
step. A properly constructed model has the ability to mimic the behavior of real system 
in a way that allows for experimentation with alternatives that may not be possible with 
the real system. Since models are dependent on the value system and purpose of use, it is 


important that one be able to determine the correctness of the model’s predictions in 
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order to validate ıt. The modeling of systems enhances our ability to comprehend and 
understand the fine differences in the interrelationships of the system elements and our 
own relationship to them. Modeling gives decision-makers the ability and opportunity to 
look at a problem or decision from several different viewpoints and perspectives. The 
last step in this process begins with the evaluation and comparison of alternatives. After 
one of the alternatives 1s selected, an implementation plan is then designed. Sage points 
out that there is a great distinction between the optimization step of the analysis process 
and the evaluation of these refined alternatives. Although there must exist more than one 
alternative for evaluation, it is important for decision-makers to attempt avoid a large 
number of biases in their evaluation and decision-making. It is the interpretation process 
of evaluation and decision-making that involves the most interactions with the previous 
two steps. 

Although there are numerous methods available to facilitate evaluation and 
decision-making, following sections and chapters will focus on the Multi-Attribute 
Utility Theory and Analytic Hierarchy Process methodologies. MAUT has been 
designed to provide a methodology for comparing and ranking alternatives that consist of 
numerous attributes or characteristics. The important attributes are identified, ordered 
and assigned a relative weight or utility by the decision-maker. The measurement 
attributes are used in the calculation of an overall utility for each alternative. 
Additionally, MAUT allows for the use of a variety of utility structures and gives 
decision-makers the ability to incorporate their attitude toward risk in the utility formula. 


(Sage, 1991) 
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The AHP application provides a method for converting subjective judgments into 
relative values in the absence of physical or statistical measures. AHP decomposes 
problems by identifying relative factors, making comparative judgments on these 
elements, and using pairwise comparison matrices to determine the relative importance 
and overall rating of the available alternatives. (Olson, 1996) 

Young (1989), believes that decision-makers can use the benefits of modeling 
within a Decision Support System (DSS) to help them deal with semi-structured decision 
problems more effectively and efficiently. Young provides the following list of potential 
benefits: 

1. To more fully understand the implication of one’s own judgments, and to 
modify judgments where they appear to inconsistent with one another or with 
what 1s known; 

2. To aid effective advocacy by means of more thorough analysis and testing of 
alternative assumptions and strategic alternatives; 

3. To identify decision problems variables to which critical outcomes appear to 
be most sensitive, so that further efforts at information gathering and analysis 
can be more effectively directed; 

4. To increase the speed and efficiency of analysis so that more alternatives can 
be examined, thereby increasing the likelihood of identifying a better strategy; 

5. To more efficiently and consistently generate, integrate, and judgmentally 
modify forecasts of outcomes needed for planning. 

Although models may not determine the single best strategy for a particular problem, 
they do provide a heuristic algorithm that almost always provides a better outcome than 
routine trial and error. 
C. DECISION SUPPORT SYSTEMS 

The evolution of DSS over the past twenty years has firmly established the 


purpose of DSS as the provider of resources to be used by decision-makers in the 


decision-making process. The DSS are available in many makes and models with 
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varying capabilities. However, all provide information. This information lends 
credibility to the final decision, but does not become the final decision. The final 
decisions that include data from the DSS and many other factors are determined by the 
decision-makers. 

Sprague and Carlson (1982) have defined DSS as: 

A class of information systems that draw on transaction processing systems and 

interacts with other parts of the overall information system to support the decision 

making activities of managers and other knowledge workers in organizations. 
Andriole (1989) agrees with this statement. As Shown in Figure 2.1, he believes that 
realm of decision support has numerous dimensions. Actual decisions are made at the 
center, while information and support activities that contribute to the evaluation of 
options and selection of the final decision reside outside in concentric circles. There are 
other definitions that are more restrictive while still others appear to be somewhat 
broader. In either case, Andniole believes that there are many similarities between 
industry, government and military decision support systems. 

Decision support includes all of the supporting data, information, activities and 
expertise required to arrive at the final selection of alternatives. However, decision 
support is not completely focused on alternative selection. Valuable decision support 


information may be provided by other decisions that have no direct connection with the 


selection of alternatives. (Andriole, 1989) 
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Figure 2.1 - The Range of Decision Support after (Andriole, 1989). 


Hogue and Watson (1983) provided that the criteria for a DSS should include the 


following principles: 


Supports but does not replace decision-making. 

Directed toward unstructured and/or semi-structured decision-making tasks. 
Data and models are organized around the decisions. 

Easy to use software interface. 

Interactive processing. 

DSS use and control is determined by the user. 

Flexible and adaptable to changes in the environment. 

Quick ad hoc DSS building capabilities. 
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A DSS has the capability to assist decision-makers with many types of decisions 
depending on system’s particular emphasis. DSS have been used to support general Jong 
range planning, strategic assessment, operational planning and control, reports and 
analysis, and general budgeting among others (Meador, 1984). As an example of this, 
USPACOM is currently developing a Theater Engagement Planning Management 
Information System (TEPMIS) to assist them in tracking planned and executed 
engagement activities conducted in their AOR. Although the list of potential applications 
continues to grow unabated, the fundamental benefits of using a DSS have remained 
relatively constant. The main benefits of DSS, as derived from numerous studies, are 
provided below in descending order of perceived usefulness. 


Provide information processing and retrieval capabilities. 
Evaluate alternatives. 

Assist in identifying problems. 

Assist in interpreting information. 

Provide real time analysis of current problems and opportunities. 
Suggest decision alternatives. 

Provide ability to ask what 1f questions. 

Manage executive time scheduling daily activities. 

Increase decision confidence. 


While this list is not all-inclusive, it 1s indicative of the capabilities desired by senior 
decision-makers when evaluating or justifying the use of such a system. (Hogue, et al., 
1985; Keen, 1981; Meador, et al., 1984; Money, et al., 1988) 

The next chapter will provide a basic introduction to the theoretical foundations 


and applications of MAUT and AHP. It is not the author's purpose to provide a complete 


analysis of these two methodologies and all of their potential applications. The 
fundamental concepts of each methodology are provided in order to provide a better 


understanding of the capabilities of the software applications. 
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II. ALTERNATIVE SELECTION 


As the problem concerns the selection of the best engagement activity to achieve 
a stated goal or objective, this section will focus on two methodologies for completing 
this task. Both of the multiple criteria decision-making aids addressed here, MAUT and 
AHP, are concerned with selecting the best choice from a given set of alternatives. 
Although MAUT is probably the more theoretically accepted approach, AHP is one of 
the more popular systems in use (Olson, 1996). Before looking at these two 
methodologies, we must address a problem both of these techniques share, one of 
determining the numerical value of a subjective judgment. 


A. QUANTIFYING SUBJECTIVE JUDGMENTS 


Nearly 50 years ago, Stevens (1951) define measurement as “the assignment of 
numbers to observations according to a set of rules.” The assignment of numerical values 
when measuring objects infers the existence of a scale of measurement. There are four 
widely accepted and commonly used measurement scales: nominal, ordinal, interval, and 
ratio. A nominal scale is simply categorizing a set of data into mutually exclusive 
subclasses. The data contained in these subclasses are counted according to frequency of 
occurrence. There is no implied order or value. In an ordinal scale, the numbers are 
assigned to the various instances of the property. This scale of measurement provides 
data about the order of the categories but does not indicate the level of the differences 
between them. Interval scales have all the properties of an ordinal scale, and in addition 


the distances between any two numbers on the scale are equally spaced. This scale of 
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measurement provides an identity, an order, and a constant unit of measure, which serves 
to indicate the exact value of an instance in a particular category. A ratio scale has all the 
characteristics of an interval scale, but 1t also has a true zero point as its origin and has 
the property of proportionality. (Rea and Parker, 1997) 

These levels of measurement are used in the process of manipulating the numbers 
into a meaningful value of the object being measured. Steven M. Anderson (1987), a 
student at the Naval Postgraduate School, proposed that there are four main traditional 
methods of quantifying subjective judgments. These methods, as presented in his thesis 
concerning a goal-programming model, are described on the following pages to show 
their potential for application to the problem at hand. 

1. NUMERICAL RATING METHOD 

The first method presented by Anderson is the numerical rating method, a very 
simple and direct method for quantifying subjective judgments. This method was first 
proposed by Stevens (1951), as a method of obtaining comparative rankings in 
psychophysical experiments. Using this method, evaluators are asked to associate rated 
items with fixed reference points by assigning them numbers, or by plotting them as 
points on a continuous number line. An example of this technique would be the 
consideration of two engagement activities and their potential effectiveness. A decision- 
maker w6uld indicate the position of these two alternatives with respect to two other 
alternatives, one that was not very effective and another highly effective activity. After 


recording all of the subjective responses, the geometric mean of the numerical estimates 
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is computed. The continuous number line representation of this example is shown in 


Figure 3.1. 


B. чун} Kata... X A... И 


less effective activity activity 1 actıvity2 highly effective activity 





Figure 3.1 - Numerical Rating Continuous Line. 

The primary advantage of the numeric rating method 1s the simplicity of its 
computation. It 1s easy to analyze the results with basic statistics and test for significant 
differences. The application of a scale relationship between these four activities results in 
interval scale data that can be transformed into any other kind of scale. The main 
disadvantage is that there 1s no natural origin for judgments, and the evaluators frequently 
disagree with the positioning of reference points. There are also no bounds on the 
interval scale; the lower bound (not effective situation) of the engagement activity 
example is set to zero, but there appears to be no upper bound. 

2. CATEGORICAL JUDGMENT METHOD 

The categorical judgment method, the second method proposed by Anderson, is a 
commonly used means of obtaining numerical ratings from subjective ratings; wherein 
evaluators assign instances to previously ordered categories. For example, staff members 
could be asked to rate the risk of a particular engagement activity according to a scale of 


high, above average, average, below average, and very low as shown in Table 3.1. This 
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example uses 25 staff members. The number of categories used can range from two to 
nine according to desires and resources of the surveyor and the skills of the evaluators 


(Fink and Kosecoff, 1985). 
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Table 3.1 - Initial Rankings of Constant Sum Method. 


The item in question is rated and the results are used to construct a matrix of the 
cumulative frequency data with n-row instances and m-column categories. Each entry of 
this matrix represents the number of evaluators who rated instance i in category j. In the 
cumulative relative frequency matrix cj, entries are the proportions of the evaluators 


rating instance ; in or below category j. This matrix is shown in Table 3.2. 


VERY 
LOW 





Table 3.2 - Constant Sum Method Cumulative Frequency Matrix. 
The elements of this matrix are considered as areas under a standard normal curve and 


are converted to the corresponding z-values. These z-values are then recorded in a z; 
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matrix (Table 3.3) consisting of n-rows and (m-/)-columns. The nghtmost column may 


be omitted for computational purposes. 


Zi HIGH ABOVE AVG BELOW 
AVG AVG 
| 






0.6406 0.5636 0.5948 0.5636 
0.6103 0.6406 0.6103 0.5319 
0.5793 0.6700 0.6255 0.5000 





Table 3.3 - Constant Sum Method Standard Normal Matnx. 


The row average, r;, and column average, c;, are calculated, and a grand average G is 


found by averaging the column averages. A column sum of squares (SSC) is computed 


as shown in Equation 3-1: 


SSC = X (c, -GY (3-1) 


For each row, the row sum of squares (SSR;) is computed using Equation 3-2: 


SSR, = > (2, 1) (3-2) 


The scale value of the instances s;, are found by solving Equation 3-3 in each row: 


SSC, 
5-С-қж | = (3-3) 


Again, the main advantage of the categorical judgment method is that it obtains 





values with interval scale properties that can be linearly transformed to any other scale. 
This method promotes the use of straightforward and uncomplicated questionnaires. 


Although it is more sophisticated than the numerical rating method, it is still relatively 
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easy to compute. The major disadvantage of this method is that the precision of its 
results is limited by the required maximum number of categories selected for use in the 
survey. (Fink and Kosecoff, 1985) However, this is not a limiting factor in the context of 
USPACOM’s engagement activity selection problem. 

3. LEAST SQUARES METHOD 

The least squares method is the third method presented by Anderson (1987). The 
least squares method is a useful and relatively simple means of obtaining scaled interval 
values from ordinal or comparative judgments. The inputs of this procedure are obtained 
by asking evaluators to do some ordinal ranking of various instances of a selected 
property. 

This method 1s begun by recording the responses of the evaluators comparing 
several items, with respect to a particular like characteristic. For example, a group of 
senior supporting staff members might be asked to rate three different theater engagement 
activities for one particular country in terms of the overall impact on the command’s 
theater engagement plan. In this instance, one staff member recommends Activity B over 
Activity C and Activity B over Activity A. The responses of this particular staff member 
are contained in the frequency matrix, fj; shown in Table 3.4. 

Since Activity B is the preferred engagement activity by the senior staff member, 
the matrix entries are made in the corresponding rows of the column B that were rated 
inferior to Activity B. In this instance, entries are made in rows A and C. Since Activity 


C was the next preferred activity, an entry is made in row A and column C. 


24 


um re —— 
a 
bui EL c 


Table 3.4 - Least Squares Method Scoring Matrix. 










The responses of all staff members are recorded in this same manner, and 
collected and recorded in another frequency matrix as shown in Table 3.5. The sum of 
the cross-diagonal elements of this matrix will be equal to the total number of the staff 
members. For this example, let's assume that there are 100 staff members. After 
reviewing each activity and evaluating it on its overall impact on the command's theater 
engagement plan, 54 staff members ranked Activity A superior to Activity C and 46 staff 
members ranked Activity C superior to Activity A. Additionally, 72 staff members 
ranked Activity A superior to Activity B and 28 staff members ranked Activity B 
superior to Activity A. To complete the matrix, 35 staff members ranked Activity B 


superior to Activity C and 65 staff members ranked Activity C superior to Activity B. 


— al өт 
m ча 
Е > se 


Table 3.5 - Least Squares Method Observed Frequency Matrix. 
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The next step is to convert the frequency matrix to a probability matrix. The 


probability matrix, p; can be calculated by using Equation 3-4: 


i= B (3-4) 
в Ул 


For the above example, the probability matrix was obtained and is shown in Table 3.6. 


Probabilities greater than 0.98 and less than 0.02 should be omitted in order to avoid 


numerical bias. 
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Table 3.6 - Least Squares Method Probability Matrix. 

The important thing to note in Table 3.6 ıs that the diagonal entries of the 
probability matrix are set equal to 0.5. The probability matrix is then converted to the 
standard normal matrix, Zy, by subtracting the mean value of 0.5 from each value of pj; 
and dividing the differences by the standard deviation of pj. The Z;; values are the 
standard normal variables corresponding to the p; values of the probability matrix. In 
regards to our example, the Z;; matrix 1s shown in Table 3.7. The least squares estimate 
of scale values s; was obtained by taking the mean of each column in the matrix Z. 

As in the previous two methods, the least squares estimate of scale values is also 


linearly transformable to other scales. Survey and questionnaire forms utilizing this 
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method along with an ordinal rating scale are simpler and require less time and effort on 
the part of the evaluators than other methods. Since this method has the advantage of 


requiring a relatively low level or ordinal assessments, the evaluators can simply list the 


Table 3.7 - Least Squares Method Standard Normal Matrix. 










instances in the order of importance regarding the compared factor. As shown in the 
example, the main disadvantage of this method is that it requires a large number of 
evaluators to produce a reasonably accurate probability matrix. Although this example 
used 100 evaluators, the number of actual experts resident on most military staffs is 
significantly smaller and would preclude the use of this method. Another issue resides in 
the cases where the evaluators do not rank all instances for various reasons, which makes 
the least squares procedure difficult to use in scale development. 

4. CONSTANT SUM METHOD 

The final method presented by Anderson is the constant sum method, a method 


developed by Comrey (1950) that quantifies subjective ratings using pairwise 
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comparisons. In this method, each activity is compared with each other by splitting 100 
points. There will be 7(n-l)/2 pairs that must be considered, and 100 points will be 
divided between each activity in accordance with an absolute ratio of the greater to the 
lesser. For example, if a staff member gives 80 points to Activity A and 20 points to 
Activity B, this indicates that Activity A is four times more important than Activity B. In 
the same manner a split of 60-40 would indicate a ratio of three to two, and 50-50 that 
two instances have the same magnitude. 

An example is used to illustrate this procedure. Suppose two staff members are 
asked to evaluate three plans on the basis of their content. Table 3.8 represents their 
respective comparison matrices where p, is the number of points is given to plan 7 when 
compared with plan j. Both staff members preferred plan A to B, plan A to C, and plan C 


to B, but the intensities of their endorsements are different. 


ee 


Table 3.8 - Constant Sum Method Comparison Matrix. 
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The next step is to construct a matrix V by averaging the p; values across evaluators as 


shown in Table 3.9. 
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Table 3.9 - Constant Sum Method Average Comparıson Matrıx. 
Another matrix J/ is formed from matrix V;. The wj; values are computed using the 


Equation 3-5 and the W matrix for this example is shown in Table 3.10: 









м; = —- (3-5) 
ji 
Activity A 1.000 0.333 0.846 








Activity B 3.000 1.000 2.333 


1.181 0.428 1.000 


Table 3.10 - Constant Sum Method W Matrix. 





As shown in Equation 3-6, the scale values can be computed by taking the и” root 


of each column product, where n is the number of instances compared in this example. 


s = y" (3-6) 


The calculation and final results of this example are shown in Table 3.11. 
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51 = [ (1.000) * (3.000) * (1.181) ] = 1.524 












s2 = [ (0.333) * (1.000) * (0.428) ] = 0.522 


s; — [ (0.846) * (2.333) * (1.000) ] = 1.254 


Table 3.11 - Constant Sum Method Scale Value Calculation. 





The constant sum method provides quantitative values that allow linear 
transformations and all arithmetic operations. The scale, therefore, presses ratio 
properties rather than interval scale properties. Its advantage over the previous methods 
lies in the fact that all of the quantitative values are on a similar ratio scale vice an 
interval scale. When using this method with a large number of instances, consistency 
becomes a problem due the potential for self-contradiction by the evaluators. 

Although each of these methods present several advantages in quantifying subjective 
judgments, none of them, in and of itself is adequate for addressing the stated problem. 
The next two sections will examine two other, somewhat more robust, methods of 
performing this task. 

В. MULTI-ATTRIBUTE UTILITY THEORY 

Decomposition of the object to be evaluated into relevant dimensions through the 
use of a hierarchical, descriptive model is the principle concept of Multi-Attnbute Utility 
Theory (MAUT). The lowest level hierarchical dimensions are considered operational. 
This indicates that they are measurable attributes of the evaluation object. Each element 
of the hierarchy is assigned a weight, which represents its relative importance in the 


overall hierarchy. Additionally, attributes at the lowest level are assigned utility 
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functions, which determine the utility for each plausible value of the attribute. 

The ngorous application of objective measurement to decision-making ıs the 
approach attempted by utility theory. This decision analysis methodology presupposes 
that better decisions will result from the rational and unbiased comparison of measurable 
attributes of performance alternatives. An alternative’s value is assumed to consist of 
measures over the criteria that contribute to its overall worth. All of these values are 
converted to some common scale of utilities. (Olson, 1996) 

Utility theory assumes that the each alternative's performance on each criterion is 
known by decision-makers. A single-measure utility function (SUF) is used to measure 
potential performance levels on each criterion. These measures are generally 
characterized, as more of a good thing is better than less. Although their measurement 
values are considered continuous, continuous scales of value are not required of 
alternatives. After identifying the single-measure utility functions, the value of each 
alternative is measured by an overall utility function. While this overall utility function 
weights each of the criteria of value, this weighting does not necessarily have to be linear. 
(Olson, 1996) 

The basic hypothesis of MAUT states that there exists a real valued function U 
defined on the set of feasible alternatives for any decision problem. Whether done 
consciously or not, it is this utility function that decision-makers wish to maximize. The 
role of the analyst is to determine the function that aggregates the criteria g1, 2,.., Zk- 


Bunn (1984) believes that utility consists of the following theoretical assumptions. 
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Structure - The choices available to the decision-maker can be sufficiently 
described by the payoff values and associated probabilities of those choices. 
This implies that the value of a choice consists of the choice's measures on 
factors of value to the decision-maker. 

Ordering - The decision-maker can express preference or indifference 
between any pair of tradeoffs. 

Reduction of Compound Prospects — Decision-makers are indifferent between 
compound prospects and their equivalent simple prospects. This implies that 
decision-makers give no value to playing the game, or gambling, or working 
in a business that is exciting because it is risky. 

Continuity - Every payoff can be considered to be a certainty equivalent for 
the prospect of some probability p of the worst outcome and the inverse 
probability (/-p). This assumption implies that severe outcomes would be 
risked at some small probability. 

Substitutability - Any prospect can be substituted for by its certainty 
equivalent. 

Transitivity of Prospects — Decision-makers can express preference or 
indifference between all pairs of prospects. This extends the assumption of 
ordering of payoffs to prospects. 

Monotonicity - For two options with the same payoffs, decision-makers 
should prefer the option with the higher probability of the better payoff. 
Cases where this is not demonstrated imply that there 1s some other factor of 
value that has not been considered. 


Additionally, Bunn indicates that there are some caveats to utility theory. As 


opposed to descriptive measures, which describe how decision-makers behave, utility 


theory is normative, describing how decision-makers should behave. One must realize 


that the behavior of many successful decision-makers is very much in violation of the 


above assumptions. Utility values are not necessarily additive in that U(A + B) may not 


be equal to U(A) + U(B). This indicates the reasonable possibility that utility functions 


can curve. It is important to note that numerical scale of utility theory 1s intended to 


order preferences, not to measure their strengths. Lastly, utilities are personal attributes 


representative of a particular decision-maker and do not carry over to others. (Olson, 
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Marshall and Oliver (1995) explain that 1t 1s seldom possible for decision-makers 
to quantify every attribute in a decision problem. In fact, they believe that as the level of 
the decision increases, the more difficult 1t becomes for one to apply some form of direct 
measurement. While it is relatively simple to preferentially order a single attribute, it is 
much more difficult for decision-makers to compare results with multiple attnbutes. A 
MAUT model where each attribute 1s assigned a separate, nonlinear utility function 
allows each attribute to be considered independently of the other attributes and is defined 
in the following paragraphs. 

The set of all possible attribute results (j = 7, 2, ...,n) is known as Rj. Assuming 
that a preference ordering is defined on each of these sets, the scalar utility from this 


vector of attributes is shown in Equation 3-7 
U(r) = Y ku (r,) (3-7) 
j=) 


where u; is a utility function defined on Ку, ky is the relative weight of attribute j, r; is an 
element of R,andr=(r,, rz,... r„.). In this particular description, along with the first two 
assumptions of basic utility theory, it is necessary to prescribe some additional 
assumptions to ensure that Equation 3-7 holds. 

The first assumption of basic utility theory states that every pair of elements in set 
R can be compared, and there exists a well-defined preference ordering of the elements. 
So if r; and r; are any two elements of set R, one should use the following symbology to 
define their relationships. Use r; > r2, if result r; is preferred to r2, r; ~ r2, if result r; 1s 


equally preferred to r2, and r; < r2, if result rz 1s preferred to r;. The second assumption 
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involves the concept of transitive preference ordering, which implies that if one prefers r; 


to r2, and r2 to r3, then one must prefer r; to r;. It is assumed that at least one of these 


hold if r/" and r/^ are any two results in R;. The “Best Result” and “Worst Result” for 


any attribute j 1s denoted as m and r,, respectively so that r- г Tas... Tn) and 


г =(Т,Г,,...› 


r,). The inclusion of these assumptions is to ensure consistency when 


comparing results. (Marshall and Oliver, 1995) 


Additionally, the following four concepts are used in determining the restrictions 


on the preference ordering and must be applied to ensure that the utility function 


decompositions hold: 


E 


2 


3. 


4. 


Preferential Independence (PI) — Given r/^ e Rj, and r/? € R;, and r; € Rj, 
every attribute j is said to be PI of attribute 1, if pe > rf) for every r. 
Mutual Preferential Independence (MPT) — Attribute j is said to be MPI of 
attribute 1, 1£7 1s PI of 1 and 1 is PI ofj. 

Utility Independence — As shown in the decision sapling in Figure 3.2, choose 
any attribute set N, and choose any three values from Ry, such that r” >r; > 
po , then choose any vector s which 1s an element of R> x R3,...,x R,. 
Attribute 1 is said to be utility independent of its complement, if the 
indifference probability p stays unchanged for every possible s. 

Mutual Utility Independence (MUI) — The subset of attributes and its 
complement are said to be MUI, if a subset of attributes is utility independent 
of its complement and vice versa. (Marshall and Oliver, 1995) 


(ri, s) 
: m, s) 
(1-р) (1, 5) 


Figure 3.2 - Diagram of Utility Independence after (Marshal and Oliver, 1995). 
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It is worth noting that the first two concepts extend to the attrıbute subsets and the 
third concept can be extended to any subset of attributes and ıts complement. The 
assumption of utility independence is much stronger than preferential independence. 
Also, attribute set N must have the MPI property if the decomposition of the utility 
function into its additive form 1s to be considered valid. However, this 1s not considered 
to be sufficient. The more complex multiplicative form is shown in Equation 3-8. 


[[h+ xk u,@,)]-1 


(г) = —— (3-8) 


Using Equation 3-8, recognize that U(r) = 0 and U(r) =]. This 1s necessary to scale 


U(r) on (0,1). Also K must satisfy Equation 3-9 


1K кк). (3-9) 


ja 

One of the important results of MAUT is that the decomposition of Equation 3-8 
only holds if the set of N attributes has the property of MUI. This result allows single- 
attribute utility functions to be combined in the same manner as multiplicative utility 
functions when MUI holds. However, certain conditions must be met conceming the 
weights 4; if the additive form of Equation 3-8 is to be considered valid. (Marshall and 
Oliver, 1995) 

In determining the k;’s, 1f we let r(7) be the vector with all attributes set to their 
worst result except the jth, which is set to its best (for an arbitrary j), as shown in 


Equation 3-10. 


UC) = К, (3-10) 
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By comparing the certainty of getting r( /) to the gamble between r and r on a decision 
sapling, the indifference probability is k;, After applying the decision-maker’s judgment 
in determining the Es, one must determine if these weights add to 1. If they do, then the 
solution to Equation 3-9 results in K = 0, Equation 3-8 simplifies to Equation 3-7 and the 
principle of additive utility holds. (Marshall and Oliver, 1995) 

If the k; s do not add to 1, but the principle of MUI holds, U can written in terms 
of the individual utility functions (w,’s) using the utility function’s multiplicative form 


(Equation 3-8). From this it can be shown that 


1. If Ik; >1,K is the unique root of Equation 3-9 ın (-1,0). 


Jal 


2. If Y k, <1, Kis the unique root of Equation 3-9 in (0,00). 


j=l 
In order to show that N has the MUI property, decision-makers must check 2"-2 subsets. 
However, decision-makers can reduce the amount of checking by using the following 
algorithm. Find an attribute that is utility independent of its complement and number it 1. 
Then check to see if each of the (7-1) pairs {1,/} is preferentially independent of its 
complement. This requires the making a total of 7 checks of which (7-1) are for the 
simpler verification of preferential independence rather than utility independence. 
(Marshall and Oliver, 1995) 

Continuing with the theme of selecting the best engagement activity, the 

following is a simplified example of the MAUT process. In this example Goodwill 
Relations, Improved Access, and Develop Coalition Partner represent the three 


engagement strategies. As shown in Table 3.12, each of these decision factors is 


36 


assigned a level of relative importance on a scale of 0 to 1. The sum of all of these 


decision factors must sum to one. 










IMPORTANCE 
0.3 
0.5 
0.2 
l 










Table 3.12 - Relative Importance of MAUT Decision Factors. 
Each of the engagement strategies will use a number of alternative engagement 
activities to achieve its objective. As before, each alternative engagement factor (Table 


3.13) is assigned a relative importance on a scale of 0 to 1. 


RELATIONS ACCESS 
ШәшҮнін T Ug 07--- 
[Exercises | 05 | 08 0 0.5 
Conference/MilTalks | 06 1.07 | 05 ___ 
High Level Visit___| 08 | 08 1. 07 | 
Foreign Military Sales | 0.6 | 06 | 06 | 









COALITION 
PARTNER 
MERERI 










Table 3.13 - Relative Importance of MAUT Alternative Factors. 
The weighted evaluation for each individual category 1s the product of each factor 
weighted and the factor evaluation of each alternative. Sum up each product (of the 
factor and factor evaluation) to obtain the total weighted evaluation for a particular 


alternative. The example shown in Table 3.14 shows the evaluation of the Port Visits 


ВИ 


alternative as ıt applıes to the strategies of Goodwill Relations, Improved Access and 
Develop Coalition Partner. 

After completing this process for each alternative to be considered under the three 
engagement strategies, compare the total weighted evaluations. The alternative that has 


the highest total weighted evaluation 1s considered the best choice. 


FACTOR NAME FACTOR WEIGHT PORT VISITS WEIGHTED 
(IMPORTANCE) FACTOR EVALUATION 
EVALUATION 


[Goodwill Relations MBE “50 == A) 024 NN 
OTAN A NOTO саш 











Table 3.14 - Evaluation of Port Visits vs. Individual Theater Engagement Plan 
Categones. 


Although the above example was simplified for ease of computation, one can see 
that multi-factor decision making can be somewhat complicated. The use of this 
methodology requires decision-makers to consider all of the various factors subjectively 
and intuitively in making the ultimate selection. After careful consideration, all of the 
important factors can be given appropriate weights and each alternative can be evaluated 
in terms of these factors. Keeny and Raiffa (1993) provide more detailed information on 
MAUT. 

C. ANALYTICAL HIERARCHY PROCESS 
The majority of the background information in this section on the Analytic 


Hierarchy Process (AHP) is drawn from the writings of Dr. Thomas L. Saaty. Saaty 
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developed the process in the early 1970”s and has continually work to improve it since 
that time. Saaty has written several books on AHP, co-authored others, and has also 
authored numerous articles concerning the value and validity of this process. 
Additionally, Saaty and Dr. Ernest H. Forman have developed a mature software version 
that has proven useful in hundreds of applications in over 27 different problem types 
including determining requirements, resource allocation, measuring performance, conflict 
resolution and choice of best policy alternative. (Saaty and Vargas, 1989) 

Specific military examples of the use of AHP and Expert Choice™ include 
vendor selection by the US Navy and the Joint Chiefs of Staff, resource allocation and 
research and development project selection by the US Army and force restructuring 
problems by the Department of Defense. 

Saaty believes that the proper arrangement of the decision factors in a hierarchic 
structure descending from an overall goal to criteria, subcriteria and alternatives is 
important. Saaty (1980) states that “a hierarchy 1s an abstraction of the structure of a 
system to study the functional interactions of its components and their impacts on the 
entire system." The decision-maker must include enough decision details to: 

represent the problem as thoroughly as possible, but not so thoroughly as to lose 

sensitivity to change in the elements; consider the environment surrounding the 

problem; identify the issues or attributes that contribute to the solution; and 
identify the participants associated with the problem (Saaty, Management 

Science, 1990). 

The arrangement of the hierarchy in this manner provides an overall view of 


situation's complex relationships and helps the decision-maker determine if the issues in 


each level are in the same category so he can make accurate comparisons. It 1s important 
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to note that these hierarchies need not be complete. An element at a particular level may 
not necessarily be an attribute of all the elements in the next subordinate level. 
Hierarchies are not the same as traditional decision trees as each separate level may 
represent different views of the same problem. (Saaty, Management Science, 1990) 

The next step in the analytic hierarchy process involves the conversion of 
subjective judgments of the decision into numerical values. As will be shown, AHP uses 
a somewhat different technique than those previously discussed. Saaty states that there 
are four major steps used in this process: 

1. Break the decision problem into hierarchical levels. 

. Collect input data by pairwise comparisons of decision elements. 

3. Using the eigenvalue method to estimate the relative weights of decision 

elements. 

4. Aggregate the relative weights at each level. 

The analytical hierarchy process 1s better suited to situations where decision- 
makers have difficulty in accurately determining the various factor weights and 
evaluations. A hierarchy of criteria and alternatives allows decision-makers to 
decompose the information contained in a decision problem. Using informed judgments 
to derive the weights and priorities, both qualitative and quantitative criteria can be 
compared. (Saaty, 1982) Again, the three most important steps in the process are to state 
the objective, define the criteria, and pick the alternatives. Patrick Harker, Decisions 
Science Department, The Wharton School, University of Pennsylvania states that “the 
overall philosophy of the AHP is to provide a solid, scientific method (the analytic part) 


to aid in the creative, artistic formulation and analysis of a decision problem (Golden, et 


al, 1989).” 
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The theoretical foundation of AHP ıs based on a set of four axıoms. The first 
axiom states that given any two alternatives į and j out of the set A, the decision-maker 
will be able to complete a pairwise comparison aj; of the alternatives under any criterion c 
from the set of criteria C . Each element a; of the pairwise comparison matrix is the 


reciprocal of aj;, or as shown in Equation 3-11. 


1 
а, === Vi,j=l,2,...,n (3-11) 


у 
а; 


The second axiom states that when comparing alternatives, the decision-maker never 
judges one to be infinitely better than the other, or aj # oo for all i, / in set 4. The third 
axiom states that the decision-maker can formulate the problem as a hierarchy. Lastly, 
the fourth axiom states that the hierarchy represents all of the alternatives and criteria that 
impact on the problem. The alternatives and criteria must represent the decision-maker’s 
intuition and should be assigned compatible priorities. (Vargas, 1990) 

To begin the process, the decision-maker must decide which of the stated 
objectives is most important as compared to the others. This process 1s known as 
pairwise comparisons and begins by laying out the overall hierarchy of the decision. This 
hierarchy serves to reveal the factors for consideration as well as various alternatives in 
the decision. Following this, pairwise comparisons are made which result in the 
determination of factor weights and factor evaluations. The alternative with highest total 


weighted score is selected as the best alternative. (Saaty and Alexander, 1989) 
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Figure 3.3 - Decision Hierarchy for Engagement Activity Selection after (Saaty 
and Vargas, 1982). 


As depicted in Figure 3.3, the hierarchical breakdown for selecting the best 
engagement activity in this example has three levels. The top level describes the overall 
decision. The second level describes the factors (goals/objectives) that are to be 
considered. The lower level of the hierarchy depicts the alternative engagement 
activities. Another level below (not shown here) the alternatives would contain the 
attributes of these alternatives or could use some sort of measurement scale to depict an 
absolute measure of effectiveness for each alternative. 

Each of the three main goals/objectives is connected to each and every one of the 


alternative engagement activities. (Saaty and Vargas, 1982) The use of hierarchies 
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allows the decision-maker to focus on each element of the decision-making process 
separately. According to Saaty (Management Science, 1990), it is more effective for the 
decision-makers to focus their judgment on a pair of elements and use a single property 
for comparison between the two without regard for the other properties or elements. 
Using pairwise comparison, two alternatives wıll be evaluated on a relative scale. The 
decision-makers can define the pairwise comparisons in any manner they choose. 

However, the prime consideration 1s that the pairwise comparisons accurately 
reflect the decision-makers’ judgment on how they want to measure their alternatives. 
Additionally, the pairwise comparisons should be listed in an ordinal fashion. Due to the 
large number of pairwise comparisons that would have to be made with large matrices, 
Saaty has recommended keeping the number of elements in any level at no more than 
nine (7+/-2) and the number of levels between three and five. It is obvious that the 
number of necessary pairwise comparisons will increase exponentially as the numbers of 
levels are increased. The pairwise comparison definitions listed in Table 3.15 from 
(Saaty and Alexander, 1989) will be used for the purpose of demonstrating this 
technique. 

Pairwise comparison in AHP is predicated on the premise that the decision-maker 
needs to evaluate a problem consisting of m criteria and n alternatives. As in the constant 
sum method, in order to evaluate m criteria, each decision-maker must make m(m-1)/2 
pairwise comparisons (where m is the number of elements on a level of the hierarchy). 
Constructing a comparison matrix, the decision-maker uses m rows and m columns to 


represent each attribute. (Saaty, 1989) 
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As stated in the axioms, AHP requires that the matrix be reciprocal. Additionally, 
the main diagonal elements of the matrix will have the value of unity (a;;= 1, Vi = 
1,2,...m). The pairwise comparison procedure is performed at every level of the 
hierarchy, with the exception of the alternatives themselves, which are the lowest level of 
the hierarchy. In other words, if there were one or more levels of subcriteria, pairwise 
comparisons in the manner described above would be performed. In the final or lowest 
level of the hierarchy, decision-makers would make a pairwise comparison of the 
alternatives one attribute at a time. Decision-makers would then evaluate each alternative 
by attribute. Ultimately, the decision-makers will have m matrices (one for each 
attribute) of size 5 x n. (Saaty, 1989) 

The pairwise comparison matrices are said to be consistent if there is a vector a of 
size n, in the case of alternatives (a would be of size m in the case of attributes), such that 
Equation 3-11 holds. Otherwise, the matrix 1s not considered to be consistent. These 
equations 1mply that 

Fo =X Vi, j,k (3-11) 
for consistency. The vector a is made unique through normalizing by dividing by its 
sum. Therefore, 

a =1 (3-12) 

If we refer to the matrix of pairwise comparisons as R; R is consistent if, and only 
if Ra = na, where a = (a;, a5, ..., a;) represents the vector of actual weights and n 
represents the number of elements. In a decision problem where some inconsistency is 


present, AHP presupposes that the decision-maker does not know a and cannot produce 
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(Intensity of Importance) 
equally to the objective 

Moderately preferred Experience and judgment 
slightly favor one activity 
over another 
Experience and judgment 
strongly favor one activity 
over another 
An activity 1s strongly 
favored and its dominance 
over the other has been 
demonstrated in practice 
The evidence favoring one 
activity over another is of 
the highest possible order of 
information 
Used when a compromise 
between judgments must be 
reached (consensus) 
Used to reflect the 
dominance first alternative 
as compared with the 
second 

























Strongly preferred 













Very strongly preferred 

















Extremely preferred 
































Intermediate values 
between 2 adjacent 
judgments 
If activity i has one of the 
above numbers assigned to 
it when compared with 
activity j, then j has the 
reciprocal value when 
compared to 7 
Ratios arising from scale 

















Reciprocals 


Table 3.15 - The AHP Point Scale for Pairwise Comparisons. 
















If consistency were forced 
by obtaining 7 numerical 
values to span the matrix 






the pairwise relative weights of matrix R accurately. In this instance, AHP solves 
Ка = А.а (3-13) 
where R’ is the matrix of observed pairwise comparisons, Amax 15 the principal eigenvalue 


of R’, and a’ is the nght eigenvector of R’ (Zahedi, 1986). This leads to an 


approximation of a, whose entries correspond to the weights of the alternatives or 
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attributes. To determine the amount of ınconsistency and determine ifthe amount of 


consistency is acceptable, Saaty developed the consistency index (CI) defined as 


cr ==") (3-14) 


(n-1) 


where n represents the number of alternatives being compared. 





Saaty explains that thıs definition of consistency goes beyond the traditional 
requirement of preference transitivity (if A 1s preferred to B and B is preferred to C, then 
A must be preferred to C) to include the actual transitive intensity with which the 
preference is expressed through the comparable alternatives. Saaty defines the concept of 
cardinal consistency in the strength of the preference as follows; if A is twice as 
preferable to B and B is three times preferable to C, then A must be six times as 
preferable to C. Saaty states that “inconsistency is a violation of proportionality which 
may or may not entail the violation of transitivity (Saaty, 1980).” Saaty further states 
that “our study of inconsistency demonstrates that it is not whether we are inconsistent on 
particular comparisons that matters, but how strongly consistency is violated in the 
numerical sense for the overall problem under study (Saaty, 1980).” If the CI < 0.10, the 
decision-maker should accept the estimate of a. Otherwise, the decision-makers should 
revise some of their judgments during the pairwise comparisons in an attempt to improve 
their consistency. (Saaty, 1980) 

Decision-makers could be forced into consistency by making just n pairwise 
comparisons in the first row of the pairwise comparison matrix. In this instance, the first 


pairwise comparison would be alternative one compared to itself, which is by definition 
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unity. Excluding this comparison, (n-1) pairwise comparisons are all that are necessary. 
In this way, decision-makers would obtain the entries for the first row of the pairwise 
comparison matrix, and define the weights based on those entries. The entries for the rest 
of the matrix could be obtained by using Equation 3-13. Again, the weights would be 
normalized to sum to one. With these weights obtained exclusively from the first row of 
the pairwise comparison matrix, every element of the matrix could be obtained. The 
resulting matrix R would be perfectly consistent. (Golden, et al, 1989) 

As indicated, AHP does not force this consistency on decision-makers. By 
requiring n(n-1) comparisons, AHP makes the vector a over-determined and allows 
inconsistencies. Therefore, the pairwise comparison matrix could very likely contain 
inconsistencies. With the presence of these inconsistencies, there 1s no exact solution for 
the vector a, such that Equation 3-11 holds for every i and j. The question is how to find 
an a that “best” fits these equations when inconsistency is present. The advantages of the 
eigenvalue method are (1) if the pairwise comparison matrix 15 consistent, Алах = п and 
(2) 1t allows evaluation of consistency by the consistency index (CT) defined above in 
Equation 3-14. (Saaty and Alexander, 1989) 

As an example, using the hierarchy in Figure 3.3, the decision-makers begin at the 
top level of the hierarchy by looking at their goal/objective and comparing the relative 
importance of Goodwill Relations, Improved Access and Coalition Partner engagement 
strategies 1n achieving this objective. Using the scale contained in Table 3.16, the next 
step will involve the determination of the relative importance of individual engagement 


activities under each of the engagement strategies. For the simplicity of this example, we 
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will assume that all three of the engagement strategies are equally important. Under the 
strategy of Goodwill Relations, the decision-makers determine that Port Visits are 
extremely preferred to Exchanges (score of 9). The decision-makers also determine that 
High Level Visits are very strongly preferred to Exchange activities (score of 7) as a 
measure of effectively conducting Goodwill Relations. Lastly, the decision-makers 


determine that Port Visits are moderately preferred to High Level Visits (score of 3). 
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Table 3.16 - Initial Pairwise Comparison of Alternatives. 

Based on the information in the preceding paragraph, the AHP technique allows 
the decision-makers to determine the evaluation factors for the three engagement 
activities under the single goal of Goodwill Relations. As shown in Table 3.17, the 
comparison of each alternative against itself results in the assigned value of 1, which 


represents equally preferred. 
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Table 3.17 - Pairwise Comparisons of Alternatives Against Themselves. 
In general, for any pairwise comparison matrix, the value of 1 will be placed 
down the diagonal from the upper left corner to the lower right corner. To finish the 


matrix, AHP makes the observation that alternative A is twice as preferred to alternative 
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B, the decision-maker can conclude that Alternative B ıs preferred only half as much as 
alternative A. Therefore, if alternative A receives a score of 2 relative to alternative B, 
then alternative B should receive a score of % when compared to alternative A. This 
same logic is used to complete the lower side of the matrix. As defined by the axioms, 
the values of the lower side of the matrix are the reciprocals of the value located directly 


diagonal across their location. The values should be as depicted in Table 3.18. 


Port Visits 


Hig сс Visits 





Table 3.18 - Pairwise Comparisons of Reciprocal Alternatives. 
The next step in the process, as shown in Table 3.19, involves the conversion of 


the fractions into decimal numbers and the summing of the column totals. 
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Table 3.19 - Conversion of Pairwise Comparison Fraction into Decimals. 
Once each column has been totaled, each number in the matrix is divided by their 
respective column totals as shown in the following table. Each column total must equal 1 
(due to computational rounding, the results shown in Table 3.20 will only approximate 1, 


but has no significant effect on the final values). 
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Table 3.20 - Normalization of Pairwise Comparisons. 
To determine the priorities of Goodwill Relations for the three alternative 
engagement activities, simply find the average of the various rows from the matrix of 
numbers as shown in Table 3.21. 


Visits 


0.6923 0.7241 0.5294 0.6486 
Hish Level Visits 0.2307 0.2413 0.4117 0.2946 
0.0769 0.0344 0.0588 0.0567 






Table 3.21 - Row Averaging of Pairwise Comparisons. 
As shown in Table 3.22, Port Visits provides the highest factor evaluation as a 
measure of effectiveness for Goodwill Relations. All of the other factor evaluations for 


the other alternatives can be determined in the same fashion. 


High Level Exchanges 
Visits 


Goodwill Relations 0.6486 0.2946 0.0567 





FACTOR 





Table 3.22 - Factor Evaluation for Goodwill Relations. 
Now that the factor evaluations have been completed the decision-makers must 
conduct one additional process to ensure that all of their responses have been consistent. 


The consistency ratio can be obtained by determining the weighted sum vector (WSV) of 
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the matrix. Using the product of factor evaluation for the first alternative (4,) from Table 
3.23 and the values of the first column (B7;, B;2, B;3) of the original pairwise comparison 
matrix (Table 3.19) initiates this process. After completing the same process for the 


second and third alternative, sum these values over the rows as shown in Equation 3-15. 


(4 X8.) (5 XB) (4, XB..) WS 


WSV = (4 ХВ„)+ (4, ХВ„)+---(4,Х8,,) is di (3-15) 


(4XB,,)+(4X8,,)+-(4,X8,), (и, 
(0.6486 (1) + (0.29463) + (0.0567 (9) 2.0427 


ЗУ = | (0.6486 )(0.3333)+ (0.2946 (1) + (0.05677) 0.9077 
(0.6486X0.1111)+(0.2946X0.1428)+(0.0567X1)| | 0.1708 


The consistency vector (CV) must be determined next. As shown in Equation 3- 
16, this step is completed by dividing the results of the weighted sum vector by the 


alternative factor evaluations contained in Table 3.22. 





WS 
ev Br. 1-Е ССА] (3-16) 
А 4 A, | 


= [3.1493 3.0811 3.0123] 


2.0427 0.9077 0.1708 
0.6486 0.2946 0.0567 


Upon completion of these two steps, decision-makers must compute the 


consistency index (CI) and lambda (A) before determining the final consistency ratio. 
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Lambda 1s simply the average value of the consistency vector and is computed using 


Equation 3-17. 


Х (2+4 +4.) (3-17) 


п 


4, — 3-1493 + 3.0811 + 3.0123 
3 


= 3.0809 


The formula for determining the CI has been previously defined as Equation 3-14. In this 


example, n = 3, for the three different alternatives being compared. 


_ (3.0809 — 3) 
ET 


Finally, the Consistency Ratio (CR) can be computed. The CR 15 equal to the CI 


= 0.0405 


divided by the Random Index (RI) as shown in Equation 3-18. The RI is a direct function 
of the number of alternatives being considered. The RI values in Table 3.23 were 


obtained by computing the eigenvalues for each n-size matrix using a sample size of 500. 
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Table 3.23 - AHP Random Index Table. 


CR == (3-18) 


К = oo = 0.0698 





For this example, CR = 0.0698. The consistency ratio tells decision-makers how 
consistent they are with their answers. A higher number indicates less consistency, 
whereas a lower indicates more consistency. As previously discussed, generally, if the 
consistency ratio is 0.10 or less, the decision-makers’ answers are relatively consistent. 
As shown above, the decision-makers’ answers in this case were fairly consistent and the 
results indicate that in this instance Port Visits are considered the best alternative for 


achieving the objective of Goodwill Relations. 


53 


D. COMPARISON OF MAUT € AHP METHODOLOGIES 

The MAUT methodology is a well-developed technique for dealing with decision 
problems containing risk and encompasses utility functions that are nonlinear, such as, 
additive linear on each variable, multiplicative and multi-linear. Also utility functions 
have been shown to be more advantageous when dealing with repetitious decision- 
making problems. However, due to rapid changes in the utility function over time, it 
must be constantly reevaluated. This requires a lot of time and effort by decision-makers 
and does not necessanly lend itself for use as a group process. 

As opposed to generating the utility function itself, the AHP methodology is 
useful in generating the functional values of a utility function. AHP gives scaled values 
that are on a ratio scale, whereas MAUT uses an interval scale. AHP provides a 
mechanism for checking on the consistency of the input data. As such, it is not necessary 
for AHP to assume consistency among the preferences. The MAUT methodology 
requires a transitive preference relation when building utility functions. 

In recent years, several articles about the disadvantages of AHP and superiority of 
MAUT over AHP have been published. Dyer (1990) points out that AHP suffers rank 
reversal (an alternative that 1s chosen as the best over a set X 1s not chosen when some 
alternative, perhaps an unimportant one, is excluded from X). He concluded that 
changing ratio scales with interval scales as in MAUT could solve this problem. Another 
method for reducing the chances of rank reversal will be addressed in the next chapter. 


Perez (1995) gave the comparison of the two methods and stated: 
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One would expect MAUT, since it requires only the construction of an interval 
scale, to be suitable for a wider range of applications than AHP. However, one 
would also expect that AHP, since it builds a ratio scale, would be more suitable 
to some situations in which the subjacent structure had a strong distributive 
component, particularly those in which the coefficients of the distribution were 
not strongly affected by changes in the set of available alternatives. 

The other shortcoming stated by Dyer (1990) 1s the scaling method of AHP. The replies 

to these criticisms by Saaty (1990), Harker and Vargas (1990), and the corresponding 

counter-replies show that no consensus on scaling has been reached. 

The focus of MAUT and AHP is to compare a defined set of alternatives against a 
value function that reflects attribute importance. Both methodologies require decision- 
makers to logically structure a complex problem and both appear to perform better when 
addressing problems with a limited number of alternatives. However, both can 
adequately deal with relatively large numbers of attrıbutes. As discussed in the next 


chapter, both methodologies have been implemented in commercial applications, which 


make 1t easier to solve and understand AHP and MAUT related decision problems. 
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IV. EXPERT CHOICE™ SOFTWARE APPLICATION AND 
EVALUATION 


Expert Choice’™ is a software application that supports decision-making by 
applying the concept of the analytic hierarchy process. This software package provides 
decision-makers with the ability to construct decision frameworks (hierarchies) from 
routine and non-routine decision problems. These frameworks logically organize the 
problem from goal to criteria to subcriteria down to the alternatives of the decision. 
Expert Choice!" was chosen for this analysis due to its availability, developmental 
maturity, problem type suitability, group applications and overall low cost. 

As previously discussed, Expert Choice™, hereafter referred to as EC, is a 
windows-based, multi-criteria decision support software tool based on the AHP 
methodology. The developers of EC, Dr. Thomas L. Saaty and Dr. Ernest R. Forman, 
believe that AHP is a powerful and comprehensive methodology capable of 
accommodating the use of empirical data and subjective judgements in aiding decision- 
makers. The AHP methodology assists decision-makers in the decision process by 
allowing them to organize and evaluate the importance of various objectives and the 
alternative solutions of a decision. In addition to facilitating all of the aspects of AHP 
discussed in the previous chapter, EC allows decision-makers to do what-if or graphical 
sensitivity analysis. These particular features allow decision-makers to quickly evaluate 
the effect of a change 1n the importance of an objective on the alternatives of choice. 

The remainder of this section will be used to describe the operation and evaluate 


utility of EC as an alternative selection software application. It is not the intent of this 
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work to recreate the user’s manual, but sufficient detail drawn from that source will be 
provided to show how the application is utilized. 
A. MODEL DEVELOPMENT 

Modeling in EC can be accomplished by three methods. In the first method 
decision-makers can simply modify previously constructed models with similar 
characteristics. These models can be stored in the Model Library. Initially, the Model 
Library contains only the sample models that are part of the software, but it can be easily 
expanded as more models are completed. In the second method, decision-makers can 
construct the model directly using the Evaluation and Choice feature. In the third and 
final method, use of the Structuring feature allows decision-makers to structure a model 
from the “top down” or the “bottom up.” Structuring will be discussed later in a separate 
section. The examples provided in this chapter use the Evaluation and Choice method. 

The first step in the creation of a decision model is the definition of the Goal 
node. The Goal node is at the top level (level 0) of the model. There must be at least two 
or more levels below this node containing the criteria and the alternatives, but more 
complex models can contain several layers of criteria (subcriteria, sub-subcriteria, etc.). 
Figure 4.1 shows a portion of graphical user interface (GUI) window of the example 
Evaluation and Choice model screen. 

Using the elements of the USPACOM decision problem, the Goal node for this 
example is Select best theater engagement activity. Goodwill, Access, and Partner are 
USAPCOM'’s three major areas of focus and represent the Criteria nodes for this goal. 


The Alternative nodes are contained in the next lower level and represent a sampling of 
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the potential alternatives that could be considered by USPACOM. The main Evaluation 
and Choice screen displays information (not shown here) in the upper right hand corner 
concerning the nodes. Using the currently selected node, decision-makers have instant 
access to information about node level, node number, global and local priority of the 
node, type of synthesis conducted on this node. Additionally, EC provides for the 
definition of each type of node up to a limit of 65 characters. The descriptions of the 
criteria and alternatives are provided in Appendix A. 

In Figure 4.1, one can also see that the priorities for each criterion are reflected as 
to their relative importance to the overall goal. The EC software will only allow one to 
view three levels of the model at a time in this particular view. However, by using the 
Sideways View function, decision-makers can see the entire model, along with the 
priorities of each criteria, subcriteria, and alternative as it applies to the goal. The model 
shown in Figure 4.2 15 the same as that shown in Figure 4.1, both of which have only 
three levels. Again, all of the priorities for each individual criteria and alternative are 
contained in Appendix A. 

When building models, decision-makers should keep in mind the relationship of 
the nodes. There are parent nodes, children nodes, and other descendant nodes. A plex is 
a branch of the model headed by a particular node, and includes all of the descendants of 
that node but not the node itself. Leaves are represented by the alternatives and always 
placed at the bottom level of hierarchy. An exception to this applies to Ratings models, 
which will be discussed later. Lastly, peer nodes are the siblings of other nodes who have 
the same parent node. This will become more important later when attempting to build 


larger, more complex models. 
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Evaluation and Choice Model Screen. 


Figure 4.1 - 
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Figure 4.2 - Sideways View of Evaluation and Choice Model Screen. 

Alternatives, also known as “leaves,” are added to the model in the same fashion 
as descendent nodes. Once decision-makers have determined that all ofthe defined 
alternatives apply to any or all of the other corresponding parent nodes, it is a simple 
matter to copy the entire set for application to the complete model. One can either copy 
the current node’s to its peers, copy the marked node’s children to the current node, or 
copy the current node’s children to form leaves below all other nodes at the bottom level 
of the model. In instances of repetitive information, EC also allows for the copying of 
complete plexes into other portions of the model. 

In addition to these basic features, EC provides a Notes feature that allows 


decision-makers an opportunity to provide detailed descriptions ofthe nodes. This 
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feature can be utilized to provide additional information on the current node such as 
explaining what information was used to develop judgments concerning this particular 
node. 

B. MODEL ASSESSMENTS 

Once the hierarchy of the model has been constructed, EC provides several 
features to accomplish analysis of the model. The first analysis to be completed is the 
assessment. The Assessment Menu will allow decision-makers assess the model with 
pairwise comparisons, data, what-if, and ratings analysis features. In making pairwise 
comparisons, decision-makers have the option of selecting from three types of 
comparison: importance, preference, or likelihood. Additionally, the actual pairwise 
comparisons can be made in several forms: verbal, graphical or numerical. 

Importance 1s normally used when comparing one criterion to another. For 
example, this model (Figure 4.3) indicates the Access alternative is moderately more 
important than the Goodwill alternative. The Preference method is more appropriate for 
comparing alternatives such as determining that Port Visits are more preferable than 
Conferences under the criterion Access. The Likelihood method is normally used when 
comparing uncertain events such as the probability of a combined exercise improving 
access. 

Of the actual] pairwise comparison methods, the Verbal comparison feature is the 
easiest to use and understand. This feature allows decision-makers to make comparisons 
of alternatives using a scale very similar to one discussed in Chapter III. As shown in 


Figure 4.3, this feature compares nodes using English language terms and may prove 


62 


very useful when comparing alternatives in terms of their social, psychological or 


political context. 
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Figure 4.3 - Verbal Comparison Screen. 
This type of comparison also contains a magnification feature, which allows decision- 
makers to make fine distinctions between two criteria that are nearly equal. This may be 
necessary in instances where decision-makers feel that indicating that a dominant element 


is twice as dominate (1.e., between Equal and Moderate) overstates the case. 
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The Numerical comparison feature may prove more useful when discussing 
alternatives from the context of economical or other measurable aspects. Figure 4.4 
contains an example in the numerical matrix form. As one can see, this representation is 
not necessarily significant when dealing with individual decisions, but would possibly be 
better suited when dealing with group decision-making. The group-decision applications 


of this software will be discussed in a subsequent section. 
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Figure 4.4 - Numerical Matrix Screen. 
The information contained in Figure 4.4 is the same as that in Figure 4.3, but now 
decision-makers can apply a numerical value to the importance of one criterion versus 


another. 
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As shown in Figure 4.5, the Graphical method of comparison is somewhat more 
intuitive, but will require some practice by decision-makers to perfect. Using a pie chart 
and a bar chart, the graphical comparison method allows decision-makers to enter 
judgments as a visual expression of preference, importance or likelihood. This feature 
may prove to be useful in instances where “fuzzy” judgments need to be made and must 


be used in cases where there are only two factors. 
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Figure 4.5 - Graphical Comparison Screen. 
Another screen shown in this example is the Preliminary Question screen. This 
screen only appears when decision-makers have chosen the Verbal method of comparison 


and only for those comparison groups for which no judgments have yet been entered or 
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previous judgements have been erased. The Preliminary Question screen allows 
decision-makers to do a quick comparison of model elements by assessing whether one 
element is more important, equally important or less important than another. It is not 
necessary for decision-makers to complete the preliminary questions for every pair before 


moving to the Assessment screen in one of the comparison modes. Also, EC gives 


decision-makers the option of skipping the preliminary questions altogether. 
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Figure 4.6 - Preliminary Questions Screen. 
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After completing the pairwise comparison process, the next step is calculating the 
results ofthe comparison mode. Figure 4.7 shows that this calculation results in the 


derived priorities ofthe criteria with respect to the parent node. 
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Figure 4.7 - Priorities/Inconsistency Screen. 

The priorities shown in Figure 4.7 indicate that Access has much more influence than the 
other two criteria in obtaining the overall goal of selecting the best engagement activity. 
These priorities are local priorities in that they sum to one. These priorities were derived 
from all of the pairwise comparisons for the current group of elements. 

Another source of important information shown in Figure 4.7 is the Inconsistency 
Ratio. As discussed in Chapter III, Saaty professes that as a general guideline, an 
inconsistency ratio greater than 0.10 is considered unacceptable. The results of this 


example indicate that it is well within the limits established by Saaty. Decision-makers 
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can address any inconsistencies by using the Reorder or Compare features, or they could 
abandon all of their previous judgments and start from scratch. The Reorder feature 
changes the order of elements in the comparison matrix according to descending priority 
and restarts the comparison process with the most important element first. As elements 
are shown in decreasing priority (left to right/top to bottom) in the Matrix mode, any 
judgement that appears out of order is probably the source of inconsistency. The 
Compare feature allows decision-makers to reenter the comparison mode they were 
previously in and then modify one or more judgments. 

The next assessment method 1s the Data method. Data entry can be used to enter 
actual data values such as costs, probabilities, periods of time or any other values. This 
feature allows decision-makers to derive priorities for the nodes in comparison from the 
data rather than from pairwise comparisons. This method allows decision-makers to 
invert priorities in instances where a higher data value is less desirable than a lower 
value. Additionally, decision-makers have the option of converting their data entries into 
pairwise comparisons. The examples provided in this work do not use this method. 

The What-If method of assessment is used to directly set the priorities using bar 
graphs. The nominal priority values are established by dividing each priority by the 
largest priority value among the group. Decision-makers have the option of changing the 
nominal values, the priority values or the bar lengths. As a prionty value is changed, the 
other priority values are changed proportionally. Once decision-makers have reach a 
final set of values, EC can easily save these values to the individual models. Again, 


examples of this method will not be provided here. 
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The final assessment method to be discussed ıs the Ratings method. The Ratings 
method combines the hierarchy and pairwise comparıson process. This method provides 
a format that allows decision-makers to shift the emphasis of analysis from one where 
alternatives are compared against each other for specific criteria to one where standards 
are established for the criteria and alternatives are compared against these standards. 
Although this method gives decision-makers the capability of analyzing a larger number 
of alternatives, 1t should only be used when decision-makers possess a thorough 
understanding of their criteria and will be able to generate meaningful scales for rating 
the criteria and subcriteria. 

In contrast to the relative measurement of the Pairwise Comparison method, the 
Ratings method employs absolute measurement. Absolute measurement allows decision- 
makers to gauge elements against an established scale or set of standards instead of 
against each other. Using this method, decision-makers can establish prioritized 
evaluation categories into which engagement activities fall. An example of these ratings 
scales could evaluate the engagement activity’s impact on the overall goal if implemented 
such as Excellent, Above Average, Average, Below Average, Poor. Another scale could 
evaluate the engagement activity’s priority with respect to the overall goal such as 
Extremely Critical, Highly Critical, Moderately Critical, Not Critical and/or Extremely 
Significant, Very Significant, Moderately Significant, Somewhat significant, Not 
Significant. A third example could evaluate the engagement activity’s risk such as High 
Risk, Significant Risk, moderate Risk, Low Risk. These ratings scales or any others 


devised by decision-makers can serve to indicate the influence of a particular alternative 
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engagement activity on the achievement of the overall goal. 

Although the criteria still undergo pairwise comparisons, the alternatives are 
compared against this previously established scale. One potential advantage of using the 
Ratings method is the inclusion of the Benefits/Costs Analysis feature. This feature of EC 
gives decision-makers a quick, simple and straightforward method for optimizing 
resource allocation decisions. 

C. SYNTHESIZING JUDGMENTS 

Now that the various assessment methods have been discussed, the next process 
concerns the synthesizing of the judgments. Synthesizing is the process of weighting and 
combining priorities throughout the model after all of the judgments are made to yield the 
final result. Synthesis determines the global weights of the model's nodes by 
multiplying each node's local priority by its parent's weight. These global weights are 
then summed to yield the overall weights. As discussed in Chapter III, the alternative 
with highest weight is the most preferred alternative. 

The Synthesis process allows decision-makers to display their results in summary 
or detail form. The summary form is shown in Figure 4.8 and indicates that Port Visits 
alternative is the most preferred engagement activity for this example. The detailed form 
of this example is contained in Appendix A. Decision-makers are also given the option 
of displaying their results in the Ideal or Distributive mode. This process does not apply 
to Ratings models as the overall weight of an alternative is obtained from the total score 


column of the spreadsheet. 
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Figure 4.8 - Synthesis Summary Results Screen. 

The Ideal mode is useful when comparing several alternatives that are very 
similar to each other. This gives decision-makers the ability to prevent the weight of the 
best alternative from being diluted by its competitors. Instead of using normalization, the 
Ideal mode divides the numerical ranks of the alternatives for each criterion by the largest 
value among them. The most preferred alternative receives a value of one. When new 
alternatives are added, they are only compared with the most preferred alternative for that 
particular criterion. 

The Distributive mode is useful in instances where the choice of the best 
alternative 1s influenced how many other alternatives exist and what those other 
alternatives actually are. The Distributive mode distributes the weights of the criteria 
among the alternatives, a process that proportionally divides the full criteria weights in a 


manner relative to the percentage of the preference of the alternatives. Discussion of this 
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mode brings up the topic of rank reversal. 

According to the developers of EC, rank reversal of the final alternatives is only 
possible ın the Distributive mode and usually occurs when other alternatives or added or 
deleted. When using relative measurement to rank alternatives, decision-makers must 
realize what happens to one alternative also affects all of the other alternatives contained 
in the hierarchy. Inserting an additional alternative that bears a close resemblance to one 
or more alternatives already under consideration essentially spreads the overall priority of 
that particular between both of those alternatives instead of just one. When an alternative 
is introduced that is a duplicate or a close copy of an existing alternative, with respect to 
the established criteria, decision-makers should revise the entire set of criteria or delete 
that particular alternative. Since the derive priorities of criteria and alternatives are 
specifically tailored for that particular decision problem, any change (adding new criteria 
or alternatives, deleting or changing others) essentially changes the problem and makes it 
anew one. A new problem that must be considered separately from the decision problem 
that previously existed. (Golden, et al, 1989) 

In either case, the results of the Distributive or Ideal method are often very similar 
to each other. The developers believe that the choice of synthesis methods depend on 
whether the goal is to select the single best alternative (Ideal/open system) or prioritize all 
of the alternatives (Distributive/closed system). 

D. SENSITIVITY ANALYSIS 
Sensitivity analysis can be used by decision-makers to investigate the sensitivity 


of the alternatives to changes in the priorities of the criteria. Starting from the Goal node, 
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sensitivity analysis shows the sensitivity of the alternatives with respect to the criteria 
immediately below the goal. This allows decision-makers to observe how the overall 
priorities of the alternatives change as the priorities of the criteria are changed. This 
analysis can be performed at lower levels of the hierarchy in models that possess more 
than three levels. 

The Sensitivity Analysis mode provides decision-makers with five graphical 
options, each of which emphasizes different aspects of the priorities of the criteria and 
alternatives. These five options include Performance, Dynamic, Gradient, 2D Plot, and 
Difference sensitivity analysis. There are two important considerations to remember 
when conducting sensitivity analysis. First, any changes іп graphical mode are 
immediately reflected in the other modes. Second, the sensitivity analysis graphs will be 
depicted differently depending on whether the type of synthesis mode 1s Ideal or 
Distributive. 

The Performance sensitivity graph depicts all of the information about how the 
alternatives behave with respect to each criterion and provides the most compact 
presentation. The Performance sensitivity graph also shows a composite sensitivity 
presentation of each alternative’s overall performance. As shown in Figure 4.9, a vertical 
line represents each of the three criteria in this example. As read from the right axis, the 
point where the alternative line intersects the particular vertical criterion line represents 
the priority received by an alternative, with respect to that criterion. The overall priority 
of the alternative is represented by its intersection with the last vertical line, which is 


labeled Overall. 
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Figure 4.9 - Initial Performance Sensitivity Analysis Screen. 
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Figure 4.10 - Performance Sensitivity Analysis Screen with Changes. 
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In this example, the sensitivity analysıs ıs done using the Ideal synthesis mode. 
The results depicted in Figure 4.9 are consistent with previous examples discussed in this 
chapter. The Access criterion still has the greatest influence of the three criteria and the 
Port Visits alternative still represents the best alternative. The results depicted in Figure 
4.10 show what occurs when decision-makers change the influence of one of the 
criterion. In this case the influence of the Access criterion was reduced to approximately 
0.300 vice 0.625 while the influence of the Goodwill and Partner criteria were increased 
proportionally. As the results indicate, the overall influence of the Port Visits alternative 
is reduced and the High Level Visits alternative becomes more significant than the 
Exercises alternative. The changes to the other alternatives appear to be insignificant in 
this case. 

The Dynamic sensitivity graph is the next method for performing sensitivity 
analysis. This graph shown in Figure 4.11, places its emphasis on the priorities of the 
criteria in the model and shows decision-makers how changing the priority of one 


criterion affects the priorities of the alternatives. 
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Figure 4.11 - Dynamic Sensitivity Analysis Screen. 


As shown in Figure 4.12, this sensitivity graph also allows decision-makers to 
look at the influence of the individual criterion on each alternative by displaying their 
components. Decision-makers can examine the influence of a criterion by simply 
dragging the bar to the right and observing how the priorities of the remaining criteria are 


decreased proportionally with respect to their original priorities. 
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Figure 4.12 - Dynamic Sensitivity Analysis Screen with Components. 


The Gradient sensitivity graph provides a linear representation of the alternatives 
against a single criterion. This sensitivity graph provides an emphasis on how the 
alternatives relate for any priority assigned to the criterion shown on the x-axis. The 
gradient sensitivity analysis lies in the points where the lines of the alternatives intersect 
with each other. This type of analysis gives decision-makers the capability to identify the 
"trade-off' points where the preferred alternative with respect to a selected criterion 


changes. Figure 4.13 provides an example of this analysis using the Access criterion. 
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Figure 4.13 - Gradient Sensitivity Analysis Screen. 
The 2D Plot sensitivity graph shows how well the alternatives perform with respect to 
any two criteria. The graph is divided into quadrants with each of the particular criterion 
being represented by the x- and y-axis. As shown in Figure 4.14, the most favorable 
alternatives as defined in the model will be depicted in the upper right quadrant, while the 
least favorable alternatives will be depicted in the lower left quadrant. The closer the 
alternatives are to the corners (upper right — best, lower left — worst), the more significant 
their impact. Alternatives depicted in the upper left and lower right indicate potential 
conflicts between the two criteria and decision-makers may need to conduct some trade- 
offs. Figure 4.14 indicates that an opportunity for trade-offs between the Exercises 


alternative and the High Level Visits alternative may exist. 
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Figure 4.14 - 2D Plot Sensitivity Analysıs Screen. 


The last sensitivity graph to be discussed is the Differences sensitivity graph. In 
this sensitivity analysis method, one alternative is fixed for comparison against any ofthe 
other alternatives. This allows decision-makers to compare criteria to see which criterion 
accounts for the prioritization ofthe alternatives. The fixed alternative is depicted on the 
left side ofthe graph. Alternatives for comparison against the fixed alternative can be 
cycled through using the tabs above the graph. Looking at Figure 4.15, the bar extends to 
the left if the alternative on the left is the best on that criterion and to the right if the 
alternative on the right is the best. This particular example depicts the differences 


between the Port Visits alternative and the Exercises alternative. The results contained 
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in Figure 14.5 show that the Port Visits alternative performs better than the Exercises 
alternative under the Goodwill and Access criteria, but performs worse than the Exercise 
alternative under the Partner criterion. Graphs of the remaining comparisons are 


contained in Appendix A. 
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Figure 14.5 - Differences Sensitivity Analysis Screen. 


E. GROUP APPLICATIONS 
1. Group Decision Making 
Group decision-making by its very nature is a somewhat contentious process, but 


it is also one that can offer some unique benefits and trade-offs. The group decision- 
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making process provides an environment for the presentation and analysis of diverse 
perspectives from multiple sources. In addition to presenting the opportunity for the 
discussion of new ideas that previously may not have received consideration, group 
decision-making can sometimes lead to a more widespread acceptance of a decision 
within an organization. Herein lies the value of a decision support tool such as Expert 
Choice'". When employed in a group setting, this software application provides a 
valuable tool for building consensus among the participants. One drawback to this 
approach 1s that there may disagreements between participants and consensus may be 
difficult to reach. However, it should be noted that group consensus may be difficult to 
reach no matter what tools, procedures, or processes are used. Reaching consensus 
should be a goal, not a requirement. (Expert Choice, 1998) 

Expert Choice!" addresses this issue by including all inputs, from each group 
member, in the model during each step of the decision-making process. Also included is 
the designation of importance, which is determined by the group, not the individual. In 
the group decision-making environment, "factors such as personal charisma, perceived 
intelligence and expertise, and size and strength of an outside constituency can make 
some participants...'more powerful' than others (Golden, et al, 1989)." Instead of 
relegating themselves to playing a lesser role in the decision-making process, EC 
manages the variability of each individual's knowledge by allowing the group to 
prioritize the relative importance of each decision-maker in the decision-making process. 


(Expert Choice, 1998) Even in military organizations where formal ranks are known and 
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relative power is easy to assess, "these weights can be used to emphasize judgmental 
contributions or to increase voting power (Golden, et al, 1989)." 

The EC software application breaks down the group decision-making process in 
three activities, Brainstorming, Structuring, and Evaluation and Choice. Decision- 
makers use the Brainstorming activity to make lists of issues, objectives, and alternatives. 
Staff members can vote on items contained in the list to determine their relevance to the 
decision problem being considered. The results of this activity provide the starting point 
of the Structuring activity. The Structuring activity can be used to group objectives and 
sub-objectives into a format that allows the decision model to be readily generated. The 
results of this activity provide the starting point of the Evaluation and Choice activity. In 
the Evaluation and Choice activity group members use the process of comparison to 
indicate their perceptions on the relative importance of each objective, sub-objective and 
alternative. As previously discussed, these comparisons are used to derive the weights of 
the objectives and alternatives, which are then synthesized to provide the results. Further 
analysis of the impact of each element upon the overall decision can be conducted 
through graphical sensitivity features discussed in the previous section. The decision 
model can be completed by using these actıvities in sequence or can be built directly in 
the Evaluation and Choice actıvity. The decision model examples provided in this 
chapter were constructed using the latter method. (Expert Choice, 1998) 

2. Questionnaire and Brainstorming 

This application is a separate feature offered by the developers of Expert 


Choice™ is not included as part of the basic or group software systems. However, it is a 
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very useful tool when decision-makers are attempting to model complex decision 
problems in a group environment. As stated ın the previous section, the Brainstorming 
activity is used to create a list of issues, objectives or alternatives. Generally, the items 
on these lists are related to each other in some fashion. An important concept in using 
this technique is get as many ideas on table as possible before the evaluation and 
prioritization process begins. The Questionnaire activity is designed to allow group 
members to provide their input on specific questions. These questions may be 
independent of each other and may be designed to acquire information on a varied and 
unrelated range of issues. Although these two activities are designed to work 
independently of each other, carefully designed questions in the Questionnaire activity 
can enhance the results of the Brainstorming activity. (Expert Choice, 1998) 

According to the developers of EC, the Brainstorming activity is a capability that 
provides a highly effective method for obtaining expert opinions about a particular 
problem and determining the initial direction of attack. This format allows group 
members the opportunity to consider a wide variety of ideas that may not have normally 
been considered. As opposed to the traditional use of brainstorming to develop creative 
ideas, EC's concept of brainstorming is intended to get all of the potential issues out in 
the open without attempting to determine their level of significance. (Expert Choice, 
1998) 

The Brainstorming activity allows the group members to list all of the potential 
items or issues individually or they may be categorized according to some particular 


characteristic. At this stage, decision-makers can use the voting feature of EC to pare 
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down the list of issues or categories to a more manageable level. This feature may also 
serve to indicate the initial popularity of certain items or issues. Jt must be noted that the 
voting conducted by group members at this stage is only intended to screen the list of 
potential issues and objectives before completing the model. Actual evaluation of the 
model will be completed in the Evaluation and Choice activity after the model 1s fully 
constructed. (Expert Choice, 1998) 

This activity allows members of the group to vote in four different ways 
depending on how the issues or objectives are defined. In the first method, each 
participating member of the group rates each item on a scale of 1 - 10 (10 - most 
important, 1 - least important, 0 - not voting). The next method of voting involves a 
simple indication of yes or no. The third method allocates a specific number of total 
votes for each participating member of the group. Any number of these votes may be 
cast for any of the items or issues under consideration. However, once group members 
have used all of their allocated votes, they may not vote on any remaining items. For 
instance, if a group member is allocated 10 votes and uses all of them on the first two 
items, then he or she cannot vote on the rest of the items being considered. The last 
method of voting consists of determining a particular number of items that each group 
member may be allowed to vote on. For example, each participating member is only 
allowed to vote 4 items. Once all four votes have been cast, this member is not allowed 
to vote for any remaining items. To ensure consistency in the results, once a voting 


method has been selected, this method should be used for voting on all of the items. 
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Once complete, the voting results are averaged for presentation. This method can also be 
used to prioritize or reduce a larger list of alternatives. (Expert Choice, 1998) 

The Questionnaire activity is similar to the Brainstorming Be in that group 
members vote on the questions under consideration. However, the Questionnaire activity 
provides for the consideration of a greater variety of ideas because it does not require that 
members of the group used the same voting method for each question. Although this 
particular feature provides for greater flexibility when developing potential questions, it 
will not allow the group to make direct comparison of votes on each of the questions 
being considered. (Expert Choice, 1998) 

The are six methods of voting included in the Questionnaire activity. The first 
two methods, Rating on a scale of 1-10 and yes/no are exactly like those methods 
contained in the Brainstorming activity. The third and fourth methods consist of 
selecting from a list of multiple choices. The third method contains a list of choices 
indicated from A-E, and the fourth method lists multiple choices numbered from 1-10. 
The fifth and sixth methods are similar to the third and fourth methods in that these 
methods consist of choosing from a list of multiple choices. However, in these two 
methods, each choice (A-E and 1-10, respectively) is assigned a weight that has been 
developed and designated by group consensus. The final results of these last two 
methods are the average of the weights of the selected choices. (Expert Choice, 1998) 

Again, the results of either or both of the Brainstorming and Questionnaire 


activities can provide the starting point of the Structuring activity. 
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3. Structuring 

In the group decision-making process, the Structuring activity allows decision- 
makers to begin the analysis of a decision problem in two different ways. The first 
method uses the alternatives or objectives generated through the use of the Brainstorming 
and Questionnaire activities. These alternatives of choice are exported directly into the 
Structuring activity for further analysis. The second method consists of entering key 
concepts and ideas into the Structuring "blackboard" as they are presented and discussed 
by the group. From here, these ideas and concepts can be organized by elements and 
assembled into a hierarchical model and exported to the Evaluation and Choice activity 
for evaluation. (Expert Choice, 1998) 

If the listing of alternatives method is chosen, the next process involves the 
identification of each alternative's pros and cons. These advantages and disadvantages 
will be translated and converted into objectives and sub-objectives or criteria and sub- 
criteria. The developers of EC believe that, 1n order to derive the objectives from the 
pros and cons, decision-makers must reconceptionalize pros and cons in value-neutral 
terms (instead of positive/negative terms) and specify them in a general form. It 1s the 
decision-maker who mentally converts the advantages and disadvantages of an issue into 
objectives. Decision-makers must determine and evaluate what each alternative's 
advantages and disadvantages means in terms of the organization's overall goals and 
objectives. Expert Choice!" can then be used to automate the process of determining 


what important objectives need to be met in order to resolve the issue. 
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The Structuring activity also allows decision-makers in a group environment to 
start the construction of their decision models from the top down or the bottom up. 
Constructing the model from the bottom up begins with the identification of alternatives 
at the bottom and progressing upward toward the overall goal. This method of model 
construction is best suited for those instances where decision-makers understand the 
alternatives better than the objectives. As previously discussed, determining the pros and 
cons of the alternatives may aid in the definition of the objectives. Once the objectives 
are categorized, they can be evaluated by the Evaluation and Choice activity. (Expert 
Choice, 1998) 

In instances of strategic planning where the objectives are better understood than 
the alternatives, top down structuring can be used. Again, the resulting hierarchy of 
objectives, sub-objectives, and alternatives can be evaluated by the Evaluation and 
Choice activity. (Expert Choice, 1998) 

F. SYSTEM/TRAINING COSTS 

Single user versions of the current release of Expert Choice™ Professional is 
relatively inexpensive at $1195.00. However, the Team Expert Choice!" 25-user bundle 
packages, which include Team Keypad, (group hardware and software), Internet/Network 
license, 25 participant licenses, 2 builders licenses, 1 year technical support and a three- 
day training seminar are somewhat more expensive at $29,995.00. The radio frequency 
keypad hardware used for group environments to facilitate the remote input of individual 
participants is available separately from numerous other sources. Expert Choice, Inc. 


offers a basic set-up consisting of 10 keypads and 1 radio frequency receiver for 
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54730.00. Extra keypads are available at the price of $285.00 each. Additionally, the 
Questionnaire & Brainstorming Keypad software application is available separately for 
the cost of $4995.00. Lastly, the three-day training seminar ($1495.00 per person) 
provides instruction on the use of the group software application, including all of the 
topics covered in this chapter and use of the remote keypad technology. In an effort to 
reduce overall training costs, this seminar could serve as a "train the trainer" function 
where only a few individuals would need to actually the attend. After completion of the 
seminar, they would then serve as organizational instructors for implementation of 


system. 
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V. CONCLUSIONS 


The two types of decision support methodologies represented in this work 
represent a portion of the current state of management science’s efforts to assist decision- 
makers in solving multicriteria decision problems. Both of these methodologies are well 
founded, in theory and in practice, and have existed for over 25 years. These decision 
support methodologies and their software applications, like all other decision support 
tools, are neither designed to replace the decision-maker nor diminish the responsibility 
for the decision made. Both of them are capable of representing a non-trivial decision 
process in an effort to expand the rational boundaries of those involved in the decision- 
making process. These and other decision support tools only serve to assist in an analysis 
of as much of the available information as the decision-maker desires. 

This decision support tool can be used by USPACOM to more effectively and 
efficiently conduct the various and multiple levels of the coordination, interaction, and 
decision making currently required by the 35 separate internal and external staff and 
support entities during the course of the Engagement Working Groups. Models 
developed by the individual country chapters can be imported and evaluated as a global 
model with respect to the CINC's overall goal. Once individual country models are 
developed and properly analyzed with respect to the appropriate priorities and weights, 
this tool can serve as a method for quantifying the impact of the most significant 


engagement activity. Additionally this decision support tool can be useful in 


9] 


comparatively assessing the value of one activity type versus another in the achievement 
of a particular goal or objective. 

As one can observe from the previous chapter, the EC software application 
provides a simple, intuitive, user-friendly method for aiding decision-makers in the 
selection of alternatives. Expert Choice™ supports decision-making by applying the 
concept of the analytic hierarchy process and provides decision-makers with the ability to 
construct decision frameworks (hierarchies) from routine and non-routine decision 
problems. These frameworks logically organize the problem from goal to criteria to 
subcriteria down to the alternatives of the decision. Although the examples provided here 
are simplistic in nature, it is easy to visualize the power of this tool if the models were 
populated with more detailed information concerning the overall goal, criteria and 
alternatives. 

Although this decision support tool, or any other currently available tool, does not 
completely address USPACOM’s requirement to assess the effectiveness of engagement 
activities proposed, planned and conducted within its AOR, Expert Choice™ does 
provide a means for quantifying subjective judgments and assisting in the decision 
process. Alone, Expert Choice’™ provides a powerful analytic tool for developing and 
narrowing down a complex and lengthy list of potential alternatives and ultimately 
selecting the best alternative with respect to the established criteria. Combined with 
Situational Influence Assessment Modeling (SIAM) or other influence modeling tools, 
Expert Choice™ should be capable of providing actual measures of effectiveness for 


individual theater engagement activities and overall theater engagement plans. 


22 


In this era of reduced budgets and closer scrutiny of how these scarce resources 
are utilized, the Cost/Benefit analysis feature of Expert Choice'” can be used by 
decision-makers and planners to analyze and maximize current resource allocations. 
Additionally, the use of Expert Choice™, along with the development and 
implementation of USPACOM’s Theater Engagement Plan Management Information 
System (TEPMIS), will provide a mechanism for comparing planned-versus-executed 
engagement activities and the comparative value assessment of one activity versus 


another in the achievement of goals and objectives. 
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APPENDIX A: EXPERT CHOICE (AHP) SYSTEM OUTPUTS AND 
GRAPHS 


The following diagrams represent the full, printable versions of the GUI screens 
contained in Section B of Chapter IV. These graphs and diagrams also include: the 
overall model hierarchy; numerical, matrix, graphic, and questionnaire views of all of the 
criteria and alternatives and their respective comparisons; synthesis of the leaf nodes with 
respect to the overall goal (ideal and distributive); and the graph outputs for the 


performance, dynamic, gradient, 2-D plot, and differences sensitivity analysis. 
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Select best theater engagement activity 








GOODWILL = \ p High Lev 
(.238) (0.182) 
GOAL ACCESS = Exercise 
(.625) (0.251) 
PARTNER = Port Vis 
(.136) (0.344) 
Сотегеп 
(0.088) 
Ехсһапде 
(0.050) 
Educatio 
(0.047) 
/ * Foreign 
(0.037) 
Abbreviation Definition 
GOAL pee c ë үр 
ACCESS Provide improved access for US 
Information exchange conferences concerning regional issues 





High Lev  Visits/Talks between high level civilians and/or senior military 

PARTNER Develop coalition partner in a particular AOR 

Port Visit in AOR conducted by ARG, CBG, or single ship 
OVERALL HIERARCHY VIEW 
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Select best theater engagement activity 


Node: 0 
Compare the relative IMPORTANCE with respect to: GOAL 
| | ACCESS | PARTNER | 
~ GOODWILL (3.0) | 2.0 | 
ACCESS | 4.0 
Row element is __ times more than column element unless enciosed in () 
Definition 
Goal Select best theater engagement activity | 
4 
GOODWILL | Foster goodwill relations among US/countries in a particular AOR | 
ACCESS | Provide improved access for US | 
| PARTNER Develop coalition partner in a particular AOR | 
CRITERION MATRIX VIEW 
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Select best theater engagement activity 


Node: 0 
Compare the relative IMPORTANCE with respect to: GOAL 


1= EQUAL 3=MODERATE 5=STRONG 7=VERY STRONG 9=EXTREME 
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Definition 
Goal Select best theater engagement activity 
GOODWILL _| Foster goodwill relations among US/countries in a particular AOR | 


ACCESS Provide improved access for US 
PARTNER Develop coalition partner in a particular AOR 


CRITERION QUESTIONNAIRE VIEW 
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Select best theater engagement activity 
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GOODWILL Foster goodwill relations among US/countries in a particular AOR | 
ACCESS Provide improved access for US | 
PARTNER 


Develop coalition partner in a particular AOR 


CRITERION GRAPHICAL VIEW 
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Select best theater engagement activity 


Node: 10000 


Compare the relative PREFERENCE with respect to: GOODWILL < GOAL 


1= EQUAL 3=MODERATE 5=STRONG 7=VERY STRONG 9=EXTREME 


нату 197817 6|51473/2 112/8145 6 7]8]9 Беса 
Wee 98785432121 815 6789] оғ 
3 |High Lev 9|8/7|6/5/4(3)2/1/2|3]4|5|6|7 |8]9 Conferen 
A неће знивнонннвнннини Exchange 
5 mente 9/87 0/5/03 21 2/9/4567 Educatio 
6 |High Lev 9|8/7(6)51413/21112|3 415167 1 Foreign 
7 |Exercise 9 8765432 1(2) 31415 67 89 Port Vis 
8 Беле 987659 32123456789 Conteren 

= 818171515) 2|3|4|5|6|7 89 9 Exchange 
10 Exercise 9876 5 (4 (41312/1|213|4|51617 8/9 Educatio 
Exercise 9876 5(4]3 21234 5617 9 Foreign 
12 |Port Vis 9/8 7/6 5(4J3 2/1/12 3 El 5 6718. 9 Conferen 
Wa Pons 98789 29487 89 ew 
‘a |Ponvis [987 65 32 1|2|9 4|S|8 7| 9 Educalc 
spaw Ви АВВ нәз 
А == E 514 (3)2|1 ASAS 9 Exchange 
17 |Conferen sss 4 HO 2(314/5(|6 7|8/9 Educatio 
| 18 | Conferen 9187 GE 4 (3) 2} 1/2 34156718 9 Foreign 
19|Баһаме 9876 53 [2 34 567 
feee [91871818 913]. 5187819 Fora 
Educa 9187 65 41821213 45 6]7 89 Fore 


Goal 
GOODWILL 





Select best theater engagement activity 


Foster goodwill relations among US/countries in a particular AOR 


High Lev Visits/Talks between high level civilians and/or senior military 
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Conduct of joint/combined military training exercises 
Port Visit in AOR conducted by ARG, CBG, or single ship 


Conferen Information exchange conferences concerning regional issues 


Exhange of senior/mid-level military officers for joint billets 


Resident attendance of joint/service/military war/staff colleges 
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Select best theater engagement activity 


Node: 10000 
Compare the relative PREFERENCE with respect to: GOODWILL < GOAL 
| Exercise | Port Vis Conferen Exchange | Educatio Foreign 
High Lev 4.0 (4.0) | 3.0 | 4.0 4.0 | 6.0 
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Select best theater engagement activity 


Node: 10000 
Compare the relative PREFERENCE with respect to: GOODWILL < GOAL 
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Abbreviation Definition 
Goal Select best theater engagement activity 
GOODWILL Foster goodwill relations among US/countries in a particular AOR 
High Lev Visits/Talks between high level civilians and/or senior military 
Exercise | Conduct of joint/combined military training exercises | 
Port Vis Port Visit in AOR conducted by ARG, CBG, or single ship 
Conferen ‘Information exchange conferences concerning regional issues | 
Exchange  Exhange of senior/mid-level military officers for joint billets | 
Educatio Resident attendance of joint/service/military war/staff colleges 


Foreign | Foreign military sales of excess military equipment 


102 


Select best theater engagement activity 
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ACCESS Provide improved access for US 
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Node: 20000 
Compare the relative PREFERENCE with respect to: ACCESS < GOAL 
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Educatio Resident attendance of joint/service/military war/staff colleges 
Foreign | Foreign military sales of excess military equipment 
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Select best theater engagement activity 
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Select best theater engagement activity 
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Select best theater engagement activity 


Node: 30000 
Compare the relative PREFERENCE with respect to: PARTNER < GOAL 
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Select best theater engagement activity 


Node: 30000 
Compare the relative PREFERENCE with respect to: PARTNER < GOAL 
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ACCESS | Provide improved access for US 
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Exercise Conduct of joint/combined military training exercises 

Foreign Foreign military sales of excess military equipment 

Develop coalition partner in a particular AOR | 

Port Visit in AOR conducted by ARG, CBG, or single ship 
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Select best theater engagement activity 
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